Proceedings of APSIPA Annual Summit and Conference 201516-19 December 2015

Diffuse noise suppression
with asynchronous microphone array based on
amplitude additivity model
Yoshikazu Murase∗ , Hironobu Chiba∗ , Nobutaka Ono† , Shigeki Miyabe∗ , Takeshi Yamada∗ , and Shoji Makino∗∗ University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8577 Japan
E-mail: {murase, chiba}@mmlab.cs.tsukuba.ac.jp, {miyabe, maki}@tara.tsukuba.ac.jp, takeshi@cs.tsukuba.ac.jp† National Institute of Informatics / SOKENDAI (The Graduate University for Advanced Studies)
2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, 101-8430, Japan, E-mail: onono@nii.ac.jp

Abstract—In this paper, we propose a method for suppressing a large number of interferences by using multichannel
amplitude analysis based on nonnegative matrix factorization(NMF) and its effective semi-supervised training. For the pointsource interference reduction of an asynchronous microphonearray, we propose amplitude-based speech enhancement in thetime-channel domain, which we call transfer-function-gain NMF.
Transfer-function-gain NMF is a robust method against drift,
which disrupts an inter-channel phase analysis. We use thismethod to suppress a large number of sources. We show that amass of interferences can be modeled by a single basis assumingthat the noise sources are sufficiently far from the microphones
and the spatial characteristics become similar to each other. Since
the blind optimization of the NMF parameters does not work wellwith merely sparse observation contaminated by the constantheavy noise, we train the diffuse noise basis in advance of thenoise suppression using a speech absent observation, which can beobtained easily using a simple voice activity detection technique.
We confirmed the effectiveness of our proposed model andsemi-supervised transfer-function-gain NMF in an experiment
simulating a target source that was surrounded by a diffuse
noise.

I. I NTRODUCTION
An asynchronous microphone array is a new frameworkthat can expand the applicability of microphone array signal
processing [1]–[6]. This framework treats simultaneous recording with independent recording devices as a multichannel
observation approach for array signal processing. The use of
asynchronous recording devices has various advantages. First,
we can easily construct a multi-channel microphone array,because we can use familiar portable recording devices such as
smartphones, voice recorders and laptop computers. Second,
we can record with high signal-to-noise ratios (SNRs) because
we can select the number and placement of the microphones
flexibly.
Unfortunately, an asynchronous microphone array also
poses problems in addition to these benefits. One of thelargest problems is the drift caused by sampling frequency
mismatches between the channels. The drift changes the timedifferences of arrival of each source according to time, anddegrades the noise suppression performance of array signal
processing based on phase analysis [3], [4]. One straightforward approach is to synchronize the recording devices [5],[6]. However, these methods suffer from high computationalcosts or constraints as regards the recording manner. Thus
for computational efficiency and unconstrained recording, we
adopt another approach to employ noise suppression in theamplitude-spectrum domain discarding the phase to achieverobustness as regards synchronization error. Togami et al.

978-988-14768-0-7©2015 APSIPA

proposed a method that uses nonnegative matrix factorization(NMF) [7] to estimate the transfer function gain (hereafterreferred to as transfer-function-gain NMF) [8]. Since NMF has
high potential for blind processing and also has considerable
freedom in terms of modifying the model and training manner,
we have proposed supervised [9] transfer-function-gain NMF
to improve the interference suppression performance.
While the methods introduced above are for the suppressionof point sources, in this paper we employ this method tosuppress a large number of sources. We show that a mass ofinterferences can be modeled by a single basis assuming thatthe noise sources are sufficiently far from the microphones andthe spatial characteristics become similar to each other. Since
the blind optimization of the NMF parameters does not work
well with only a sparse observation contaminated by a constantheavy noise, we train the diffuse noise basis in advance of thenoise suppression using the speech absent observation, which
can be obtained easily using a simple voice activity detection
technique. We confirmed the effectiveness of our proposedmodel and semi-supervised transfer-function-gain NMF in anexperiment simulating a target source surrounded by a diffuse
noise.
II. O BSERVED SIGNAL MODEL OF DIFFUSE NOISE
A. Problem statement
Before discussing our proposed asynchronous observed
signal model, let us begin with the signal modeling of synchronized observation assuming that one target source andK (K ≫ M ) noise sources are recorded by M microphones.
We indicate the components of the target and noise signals
with the superscript symbols S(Signal ) and N(Noise), respectively. Assuming that a simultaneous mixing model holds
true in the time-frequency domain, the observed signal can be
expressed by the sum of the target and noise signals:
X(ω) = XS (ω) + XN (ω),(1)

where, X(ω), XS (ω) and XN (ω) constitute a matrix with aSsize M × N and have the complex-values Xmn (ω), Xmn(ω)
N
and Xmn (ω), respectively, in the (m, n) element. ω and N
represent the frequency index and the number of time frames,
respectively. Here, XS (ω) is expressed by
XS (ω) = AS (ω)SS (ω),(2)

where, A (ω) is a column vector with a size M × 1 andthe element of the vector ASm (ω) shows the transfer function
from the target source to the mth microphone. SS (ω) is a row

599

S

APSIPA ASC 2015

Proceedings of APSIPA Annual Summit and Conference 201516-19 December 2015

k=1

where, ANk (ω) is a column vector with a size M × 1 andkthe element of the vector ANm (ω) shows the transfer function
from the kth noise source to the mth microphone. SNk (ω) isa row vector with a size 1 × N and the element of the vector
SnNk (ω) shows the time-frequency component of the kth noise
source in the nth time frame.
The above model is valid for the case with the synchronous
microphone array but invalid for that with the asynchronous
microphone array. This is because, Xmn (ω) is affected by the
sampling frequency mismatch among the recording devices asfollows [10].
Ymn (ω) ≈ Xmn (ω)exp(−ȷωϵm n),√(4)

where, ȷ and ϵm represent an imaginary number (ȷ = −1)
and the value of the drift from the 1st microphone ϵ1 = 0,
respectively. For this reason, the asynchronous microphonearray causes phase drift and the process becomes complicated.
Therefore, we show a model that works without the phase
information, and which can even be applied to asynchronousrecording.
B. Mixing model in amplitude domain
Assuming the additivity of the amplitude in the frequency
domain, the mixing model can be expressed by the productsum of the amplitude spectrum omitting the phase;|X(ω)| ≈ |XS (ω)| + |XN (ω)|.

(5)

Moreover, the target source |XS (ω)| and the noise source|XN (ω)| in the amplitude domain are expressed by|XS (ω)|= |AS (ω)||SS (ω)|,
K∑
N|X (ω)| ≈|ANk (ω)||SNk (ω)|,

(6)
(7)

Frequency
Channelvector with a size 1 × N and the element of the vector SnS (ω)
shows the time-frequency component of the target source inthe nth time frame.
In addition, the noise signal XN (ω) can be expressed by
the sum of the K noise sources as below.
K∑
XN (ω) =ANk (ω)SNk (ω)(3)

Fig. 1. Channel time-frequency domain representation of observed signals foreach frequency bin

amplitude are expressed as follows.
|X̂N (ω)| ≈ |ÂN ||ŜN |,|ÂN (ω)| ≈ |AN1 (ω)| ≈ . . . ≈ |ANK (ω)|,
K∑|ŜN (ω)| ≈|SNk (ω)|,

978-988-14768-0-7©2015 APSIPA

(8)
(9)
(10)

k=1

where, |ÂN (ω)| is a column vector with a size M × 1 andthe element of the vector |ÂNm (ω)| shows the transfer function
gain from the diffuse noise to the mth microphone. |ŜN (ω)|
is a row vector with a size 1 × N and the element of thevector |ŜnN (ω)| shows the absolute value of the amplitudeof the diffuse noise in the nth time frame. Accordingly, theobserved signal model in the amplitude domain in a diffuse
noise environment can be expressed by|X(ω)| ≈|A(ω)| ≈|A(ω)||S(ω)|,][|AS (ω)| |ÂN (ω)| ,[ S]|S (ω)||S(ω)| ≈.
|ŜN (ω)|

(11)
(12)
(13)

The purpose of this study is to suppress the diffuse noise signal
and enhance the target signal. In particular, we assume thatS
the absolute value of AS
1 (ω) is at its highest in Aj (ω), j =1, . . . , M because the 1st microphone can be placed closest
to the target source utilizing the flexibility of asynchronousrecording. Thus the diffuse noise suppression signal is given by
suppressing the diffuse noise from the highest SNR signal withthe first microphone. In the following, all the modeling andprocessing can be carried out at each frequency bin. Therefore,
we omit ω for simplicity.

k=1

where, |AS (ω)| and |ANk (ω)| show the transfer function
gain of the target and noise source, respectively. |SS (ω)|
and |SNk (ω)| show the absolute values of the amplitudeof the target and noise source, respectively. This model isthe conventional observed signal model [9] for the transferfunction-gain NMF and is only valid if the numbers of target
and noise sources are already known. However, if the noise
sources become diffuse noise that contains an indefinite number of noises, we cannot suppress the diffuse noise using the
NMF described below with the conventional observed signal
model. Therefore, we propose the observed signal model forsuppressing the diffuse noise.
If the K noise sources act like the diffuse noise sources that
arrive from far off and are scattered, the average energies of
the noise sources in the amplitude domain are similar [11].
Therefore, we assume that the transfer function gain vectorsof noise sources can be expressed as one common transfer
function gain vector. And then we assume that the observed
signal, the transfer function gain and the absolute value of the

SourceTime frame

III. D IFFUSE NOISE SUPPRESSION WITH NMF
A. Diffuse noise suppression with transfer-function-gain NMF
In this section, we describe diffuse noise suppression withthe proposed observed signal model in the amplitude domain,which uses NMF to estimate the parameters of the model. The
parameterization of the NMF is shown in Fig. 1: Typically, thedecomposition of NMF in audio and acoustic signal processing[12]–[14] such as decomposition into spectral patterns andactivations, is not used. NMF approximates a nonnegative
matrix as two low rank nonnegative matrices as follows.
|X| ≈ X̃ = ÃS̃,(14)

where, the tilde represents the decorated matrices or elementsthat are the values estimated by NMF in the amplitude domain.
Here the distance measure between |X| and ÃS̃ can becustomized for the task by choosing from a variety of functions
that NMF can minimize. A low-rank approximation with aminimum distance means that the solution of S̃ will be sparsedue to the non-negative constraint. As a result, the estimation

600

APSIPA ASC 2015

Proceedings of APSIPA Annual Summit and Conference 201516-19 December 2015of the source amplitude is accompanied by the identificationof the transfer function gain and the source activation. In otherwords, Ã and S̃ estimate |A| and |S|.
We cannot decide the size of the initial matrices of NMF
using the conventional observed signal model because thenumber of sources is unknown in a diffuse noise environment.
Even if we can set the initial matrices, we cannot expect to estimate NMF because the sources outnumber the microphones.
By contrast, using the proposed observed signal model, we candecide the size of the initial matrices for NMF because the only
sources are the target and diffuse noise. Moreover, if we use
more than two microphones, we can avoid underdeterminationand estimate the parameters with NMF.
The procedure for suppressing diffuse noise by transferfunction-gain NMF using the proposed observed signal modelis as follows. First, we estimate the parameters Ã and S̃
with NMF. After that, a diffuse noise suppression signal Y
is obtained with the observed signal X1n of the m = 1
microphone and a Wiener mask as
)2
(
S
ÃS
1 S̃n
Y = X1n (
(15)
)2 (
)2 .
N S̃ N
S
ÃS
S̃
+
Ã
1 n
1 n

volves learning the basis vectors of the transfer function gain.
The transfer function gain vectors ÃS and ÃN are obtainedby pre-estimations with NMF. The pre-estimations require
two observations containing only the single source durationof the target and the diffuse noise source, respectively. The
second step is to estimate the activation vectors S̃S and S̃Nof the observed signal. The activation vectors S̃S and S̃N are
obtained by updating (19). Meanwhile, the transfer function
gain vectors ÃS and ÃN are fixed to the trained transfer
function gain vectors. Finally, we suppress the diffuse noisewith (15) derived from the trained transfer function gain andestimated activation.

This Wiener filtering reduces the error in S̃ caused by themodel mismatch of the linear modeling in the amplitude
spectrum domain. We set the initial value of ÃS as{
1 − (M − 1)α (m = 1)
S
(16)
Ãm =
α
(m ̸= 1)
because the assumed absolute value of AS
1 is the highest inASj , j = 1, . . . , M . Here, α is an arbitrary positive numberthat satisfies 0 < α < 1/(M − 1). Also, we set the initialvalue of S̃N as
1
ÃN(m = 1, . . . , M ),
(17)
m =
M
due to the diffusibility.
If the numbers of microphones and sources are similar, theestimation accuracy of NMF is degraded and higher performance cannot be expected. Thus, some methods for appending
restrictions to the NMF have been proposed. Therefore, we
propose a method for applying the observed signal modelfor the diffuse noise to the conventional method. Moreover,
we propose the semi-supervised transfer function gain NMF
training of only the diffuse noise transfer-function gain.
B. Supervised transfer-function-gain NMF
Supervised transfer-function-gain NMF is a noise suppression method that can estimate activation closer to the optimum
solution by using the trained transfer function gain. We employ
I-divergence as the distance regulation and each parameter isestimated with the following multiplicative update rules.
i∑|Xmn |S̃nÃim ← Ãim∑
S̃ni ← S̃ni

n ÃS S̃ S +ÃN S̃ N

m n
m n
, (i = S, N)
∑
i
S̃
n
n

|Xmn |Ãimm ÃS S̃ S +ÃN S̃ N

∑m n i m n . (i = S, N)
m Ãm

(18)

(19)

The procedure for suppressing diffuse noise by supervisedtransfer-function-gain NMF is as follows. The first step in-

978-988-14768-0-7©2015 APSIPA

C. Semi-supervised transfer-function-gain NMF
In this section, we propose using the semi-supervisedtransfer-function-gain NMF, which trains only the transfer
function gain vector of diffuse noise. Supervised transferfunction-gain NMF requires the single source duration of
all sources. However, with a constantly noisy space such as
outside, it is difficult to obtain the single source durationsof the target source. Therefore, we propose semi-supervisedtransfer-function-gain NMF where we apply semi-supervised
NMF in the time-frequency domain [16] to the transferfunction-gain NMF. In particular, the transfer function gain
vector of the diffuse noise ÃN is only trained by the diffuse
noise duration. After that, ÃN is fixed when (18) and (19) are
updated.
D. Penalized transfer-function-gain NMF
Penalized transfer-function-gain NMF is a noise suppression
method that introduces a sparse constraint into the source
activation. The objective function is as follows.
J (|X|, ÃS̃) = DI (|X||ÃS̃) + λg(S̃),(20)

where g(S̃) is a function for measuring the spatial sparsenessof S̃ and we employ L0.5 norm. λ shows the non-negative
weight and it is calibrated in conformance with to the observed signal. The penalized multiplicative update rule with
I-divergence is given byÃim ← Ãimi
Σn|Xmn |S̃niiÃm S̃nΣn S̃ni|X

S̃ni ← S̃ni

, (i = S, N)

(21)

|Ãi

k
Σm Ãmni S̃ imnΣm Ãim +λ∇g(S̃ni ), (i = S, N)(22)

where the ∇g(S̃ni ) shows the gradient of g(S̃ni ). Penalized
transfer-function-gain NMF suppresses the diffuse noise fromthe observed signal using parameters estimated with the above
update rules.
IV. E XPERIMENTAL EVALUATION
A. Experimental condition
In this experiment, to confirm the effectiveness of the
proposed model for the diffuse noise, we investigated the performance of methods where the proposed model was applied
to conventional transfer-function-gain NMF. Moreover, weinvestigated the effectiveness of the semi-supervised transferfunction-gain NMF. Table 1 shows the experimental conditions. Each signal from each source to each microphone wasgiven as a convolutive mixture of clean speech and impulse
responses by using the image method [17]. To obtain the601

APSIPA ASC 2015

Proceedings of APSIPA Annual Summit and Conference 201516-19 December 2015
TABLE IE XPERIMENTAL CONDITIONS

16,000 Hz
4096 samples
2048 samples
10 sec

10 sec

I-divergence
0.25
200
0.3 sec
0, 5 dB









SDR [dB]

Patt. 3
Mic 1, 4, 7
Mic 2, 5, 8
Mic 3, 6, 9

B. Results of evaluation experiment
Figure 3 shows the experimental results. (a) and (b) show
the diffuse noise suppression performance (SNR = 0 dB)
recorded with synchronous and asynchronous microphonearrays, respectively. (c) and (d) show the diffuse noise suppression performance (SNR = 5 dB) recorded by synchronousand asynchronous microphone arrays, respectively. In all thenoise suppression methods employing our proposed observed
signal model, the SDRs were higher than the unprocessedvalues. This result shows that these methods can suppress
diffuse noise. The performance of semi-supervised transferfunction-gain NMF is better than that of penalized transferfunction-gain NMF under all conditions. When SNR = 0
dB, the performance of semi-supervised transfer-function-gain

6
5
4
3
2
1
0

11
10
9
8
7
6
5

SNMF

SDR [dB]

Unprocessed

impulse response, we assumed all the microphones to be omnidirectional. Moreover, the experiment was conducted with 3,
6 and 9 microphones. The observed signals, recorded with theasynchronous microphone array, were given by the resamplingof the synchronous data according to Table II. Figure 2 shows
the arrangement of the microphones and sources. Here, we
regard the noise sources as diffuse noise by keeping the noise
sources at a distance from the microphones. The signal-todistortion ratio (SDR) is used as the evaluation score [18].
The SDR evaluates the distortion of a noise suppressed signal.
Higher SDR values indicate better noise suppression. We
calculated the evaluation scores of the unprocessed observation (Unprocessed) and of three methods that employed our
proposed observed signal model, 1) the supervised transferfunction-gain NMF (SNMF), 2) the semi-supervised transferfunction-gain NMF (SSNMF) and 3) the penalized transferfunction-gain NMF (PNMF). Here, the supervised transferfunction-gain NMF trained the transfer function gain vectorsof the target and diffuse noise source assuming that we
could obtain single-source sections containing only one source.
In particular, the scores of supervised transfer-function-gain
NMF showed the marginal performance of transfer-functiongain NMF. In addition, penalized transfer-function-gain NMF
provided the optimum value for all values of parameter λunder each condition.

978-988-14768-0-7©2015 APSIPAFig. 2. Arrangement of speakers and microphones

SDR [dB]

16,000 Hz
16,001 Hz
16,002 Hz

Patt. 2
Mic 1, 4
Mic 2, 5
Mic 3, 6

TABLE IIS AMPLING FREQUENCIES ON EACH PATTERN

Patt. 1
Mic 1
Mic 2
Mic 3



    

SSNMF

6
5
4
3
2
1
0

3
6
9
Number of microphones(a)SDR [dB]

Sampling frequency
for synchronous recording
Frame length
Frame shift
Signal length for evaluation
Signal length for supervisedand semi-supervised NMF training
Divergenceα (initialization parameter)
Number of NMF iterations
Reverberation time
Signal to diffuse noise ratio

3
6
9
Number of microphones
(c)

11
10
9
8
7
6
5

PNMF

3
6
9
Number of microphones
(b)

3
6
9
Number of microphones(d)Fig. 3. SDRs with (a) synchronous recording SNR = 0 dB, (b) asynchronousrecording SNR = 0 dB, (c) synchronous recording SNR = 5 dB and (d)asynchronous recording SNR = 5 dB

NMF was as good as that of supervised transfer-function-gain
NMF. Moreover, a comparison of (a) with (b) and (c) with(d) shows there is little difference between the performanceobtained with the synchronous and asynchronous arrays. Thus,
these methods were robust against the drift caused by sampling
frequency mismatches.
V. C ONCLUSION
In this research, we proposed an observed signal model
for suppressing diffuse noise. Moreover, we proposed semisupervised transfer-function-gain NMF, which can suppress
the diffuse noise in a constantly noisy environment. As a result,
all the methods that employed the proposed observed model
could suppress diffuse noise. Furthermore, semi-supervisedtransfer-function-gain NMF outperformed performance thanthe penalized transfer-function-gain NMF.
VI. ACKNOWLEDGMENT
This work was supported by Grant-in-Aid for Scientific
Research (B) (Japan Society for the Promotion of Science(JSPS) KAKENHI Grant Number 25280069).

602

APSIPA ASC 2015

Proceedings of APSIPA Annual Summit and Conference 201516-19 December 2015

R EFERENCES[1] M. Souden, K. Kinoshita, M. Delcroix and T. Nakatani, “Distributed
microphone array processing for speech source separation with classifier
fusion,” Proc. MLSP, 2012.
[2] A. Bertrand, “Applications and trends in wireless acoustic sensor networks: a signal processing perspective,” Proc. SCVT, 2011.
[3] E. Robledo-Arnuncio, T. S. Wada and B.-H.Juang, “On dealing withsampling rate mismatches in blind source separation and acoustic echo
cancellation,” Proc. WASPAA, pp. 34–37, 2007.
[4] Z. Liu, “Sound source separation with distributed microphone arrays in
the presence of clock synchronization errors,” Proc. IWAENC, 2008.
[5] S. Miyabe, N. Ono and S. Makino, “Blind compensation of inter-channel
sampling frequency mismatch with maximum likelihood estimation in
STFT domain, ” Proc. ICASSP, pp. 674-678, 2013.
[6] R. Sakanashi, N. Ono, S. Miyabe, T. Yamada and S. Makino, “Speechenhancement with ad-hoc microphone array using single source activity,” Proc. APSIPA, pp. 1-6, 2013.
[7] D. D. Lee and H. S. Seung, “Algorithms for nonnegative matrix factorization, Proc. NIPS, pp. 556-562, 2000.
[8] M. Togami, Y. Kawaguchi, H. Kokubo and Y. Obuchi, “Acoustic echo
suppressor with multichannel semi-blind non-negative matrix factorization, ” Proc. APSIPA, pp. 522-525, 2010.
[9] H. Chiba, N. Ono, S. Miyabe, Y. Takahashi, T. Yamada and S. Makino,“Amplitude-based speech enhancement with nonnegative matrix factorization for asynchronous distributed recording, ” Proc. IWAENC, pp. 204208, Sept. 2014.
[10] S. Miyabe, N. Ono and S. Makino, “Blind compensation of interchannel
sampling frequency mismatch for ad hoc microphone array based on
maximum likelihood estimation, ” Signal Processing, vol. 107, pp. 185196, Feb. 2015.
[11] N. Ito, H. Shimizu, N. Ono and S. Sagayama, “Diffuse noise suppression
using crystal-shaped microphone arrays, ” IEEE Trans. on Audio, Speech& Language Processing, vol. 19, no. 7, pp. 2101-2110, 2011.
[12] P. Smaragdis, and J. C. Brown, “Non-negative matrix factorization formusic transcription, ” Proc. WASPAA, pp. 177-180, 2003.
[13] T. Virtanen, “Monaural sound source separation by nonnegative matrix
factorization with temporal continuity and sparseness criteria, ” IEEE
Trans. on Audio, Speech & Language Processing, vol. 15, no. 3, pp.
1066-1074, 2007.
[14] C. Févotte, N. Bertin and J.-L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis,
Neural Computation, vol. 21, no. 3, pp. 793-830, 2009.
[15] I. A. McCown and H. Bourlard, “Microphone array post-filter basedon noise field coherence, ” IEEE Trans. on Audio, Speech & Language
Processing, vol. 11, no. 6, pp. 709-716, 2003.
[16] P. Smaragdis, B. Raj and M. Shashanka, “Supervised and semisupervised separation of sounds from single-channel mixtures, ” Proc.
ICA, pp. 414-421, 2007.
[17] E. A. P. Habets, “Room impulse response (RIR) generator,” Available at: http:/www.audiolabs-erlangen.de/fau/professor/habets/software/
rir-generator Oct. 2008.
[18] E. Vincent, R. Gribonval and C. Févotte, “Performance measurementin blind audio source separation, ” IEEE Trans. on Audio, Speech &Language Processing, vol. 14, no. 4, pp. 1462-1469, 2006.

978-988-14768-0-7©2015 APSIPA

603

APSIPA ASC 2015