2017 25th European Signal Processing Conference (EUSIPCO)

Multiple Far Noise Suppression in a Real
Environment Using Transfer-Function-Gain NMF
Yutaro Matsui∗ , Shoji Makino∗ , Nobutaka Ono† , Takeshi Yamada∗∗ University of Tsukuba, Email: y.matsui@mmlab.cs.tsukuba.ac.jp, maki@tara.tsukuba.ac.jp, takeshi@cs.tsukuba.ac.jp† National Institute of Informatics / SOKENDAI, Email: onono@nii.ac.jp
Abstract—Conventional noise suppression methods based onarray signal processing use phase information and control the
directivity of noises. However, such methods can hardly suppress
so-called background noise, whose arrival direction cannot be
specified. Thus, multiple far noise suppression based on transferfunction-gain non-negative matrix factorization (NMF) has been
proposed as a method that can suppress such backgroundnoise. Its effectiveness has been confirmed by an experimental
simulation using convolutional mixtures; however, it has not beenverified that it is practical in a real environment. Thus, in thispaper, we examine the performance of this method by recording
a target and multiple far noises with asynchronous microphones
in a real environment. We confirm that this method can suppress
far noises in a real environment with diverse distances between
microphones and interference sources.

The effectiveness of noise suppression based on transferfunction-gain NMF has been confirmed by a computer simulation using convolutional mixtures [7]; however, it has notbeen verified that it is practical in a real environment. Thus,in this paper, we verify the effectiveness of this method in areal environment by recording multiple noises asynchronously
and employing this method of noise suppression. Although
the computer simulation [7] realized asynchronization by
artifically generating a sampling frequency mismatch, werecorded voices with separate IC recorders to naturally realizea sampling frequency mismatch. Our experiment evaluated thepractical performance of the proposed method.
II. OBSERVED SIGNAL MODEL OF FAR NOISE

I. INTRODUCTION
Many noise suppression methods based on array signal processing have been proposed to improve the quality of recorded
speech [1]–[4]. They suppress noises by using phase information and controlling the directivity of noises. However, suchmethods have the following issues. First, their performance isdegraded if they are employed for asynchronous recording.
Asynchronous recording has recently attracted considerable
interest because it is possible to easily and flexibly construct a multichannel microphone array using familiar portable
recording devices such as mobile phones and voice recorders.
However, in asynchronous recording, drifts are caused by differences in the recording start time or the sampling frequency
mismatch among the channels. The drifts change the timedifference of arrival of each source over time and degrade
the performance of array signal processing based on phaseanalysis [5], [6]. Next, they can hardly suppress so-calledbackground noise, whose arrival direction cannot be specified.
These methods also use a number of sound sources and thearrival directions of noises. Therefore, they cannot suppress the
background noises if their arrival directions and the numberof sources are unknown.
Recently, a new method of noise suppression has been
proposed [7] that solves the above issues. In this method, phase
information is not necessary because array signal processing
is employed in the amplitude domain. Also, by assuming thatthe background noises arrive from far away, such noises aremodeled by a single basis. In other words, it is assumed thatthe observed signal is composed of two sources, a target source
and a mixed noise source. Then transfer-function-gain nonnegative matrix factorization (NMF) is employed with thismixing model. Note that the amplitude-spectrum beamformer[8] is also a method of noise suppression in the amplitude domain. Although the amplitude-spectrum beamformer requires
advance learning for all voices contained in an observed signal
using each single sound source, transfer-function-gain NMF
does not require such learning. Therefore, this method is
expected to be more suitable for background noises than theamplitude-spectrum beamformer.

ISBN 978-0-9928626-7-1 © EURASIP 2017

A. Amptitude-based mixing model
In this section, we describe the signal modeling of asynchronous observed signals. Before describing asynchronous
observation, we begin with the synchronous observation ofa target source and K noise sources by M microphones.
The difference between synchronization and asynchronization
is whether or not phase drifts are caused. In asynchronousrecording, phase drifts are caused among the observed signals
by the difference in the recording start time or the sampling
frequency mismatch among the recording devices.
Then, the observed signals in the time-frequency domain
can be expressed by (1) as the sum of the target signal X S (ω)
and the noise signal X I (ω).
X(ω) = X S (ω) + X I (ω)(1)

X(ω), X S (ω) and X I (ω) are matrices of size M × N andS
I
have the complex values Xmn (ω), Xmn(ω) and Xmn(ω),
respectively, for their (m, n) elements. ω and N represent thefrequency index and the number of time frames, respectively.
We indicate the components of the target signal and the noise
signals with the superscripts S (signal) and I (interference),
respectively. Then, X S (ω) and X I (ω) are expressed by
X S (ω) = aS (ω)sS (ω),
K∑
X I (ω) =aIk (ω)sIk (ω),

(2)
(3)

k=1

where aS (ω) and aIk (ω) are M × 1 vectors and the elements
IkaSm (ω) and am (ω) describe the transfer function from thetarget source or noise source k to microphone m, respectively.
sS (ω) and sIk (ω) are 1 × N vectors and the elements sSn (ω)
and sInk (ω) describe the time-frequency components of thetarget source or noise source k in time frame n, respectively.
The above model is valid for a synchronous microphone arraybut invalid for an asynchronous microphone array, becauseam (ω) is affected by phase drifts and can be time-varying.

2378

2017 25th European Signal Processing Conference (EUSIPCO)෩𝑨

channel𝑿

𝑺෨gain âI and the absolute value of the amplitude ŝI of such
far noises can be expressed as≈time frame

basis

Although an asynchronous recording has phase drifts, we
can assume that the amplitudes of the transfer function aretime-invariant if the phase drifts are sufficiently less than the
short-time Fourier transform (STFT) frame width. Therefore,
assuming the additivity of the amplitudes in the frequency
domain, the mixing model can be expressed by the productsum of the amplitude spectrum omitting the phase as follows:(4)

Such a mixing model in the power or amplitude domain isfrequently assumed when NMF is employed [9]. Moreover,
the target source X S (ω) and the noise sources X I (ω) in
the amplitude spectrum are expressed by
X S (ω) = aS (ω) sS (ω) .
X I (ω) ≈K∑

aIk (ω) sIk (ω) .

(5)
(6)

k=1

aS (ω) and aIk (ω) are the transfer function gains of thetarget and the noise source, respectively. sS (ω) and sIk (ω)
are the absolute values of the amplitude of the target and thenoise source, respectively. As described above, we introduce
the mixing model in the amplitude domain. ã(ω) and s̃(ω) are
estimated from |X(ω)| by employing transfer-function-gain
NMF [10] in the time-channel domain as shown in Fig. 1.
Note that such a mixing model in the power or amplitude
domain has been frequently assumed in the NMF context. It
is only valid to apply transfer-function-gain NMF to the model
when the numbers of targets and noises are known. However,the number of noise sources is assumed to be unknown andthus we cannot suppress the noises. Therefore, in the next
section we describe the mixing model [7] used to suppress
such background noises.
B. Mixing model of far noise for suppressing backgroundnoisesIn this section, we describe the mixing model [7] used tosuppress the background noises. It is based on the idea thatthe background noises are composed of multiple far noise
sources. Here, we assume that K background noises arrive
from much farther than the target and are scattered. In thiscase, the average energies of the observed noise sources are
typically similar in a diffuse noise field [11]. Therefore, we
assume that the transfer function gains of all the noise sources
are similar if they are far away, and the transfer function gain
vectors of the noise sources can be expressed by a common
vector. Thus, the observed signal X̂ I , the transfer function

ISBN 978-0-9928626-7-1 © EURASIP 2017(7)âI (ω) ≈ aI1 (ω) ≈ · · · ≈ aIK (ω) ,

(8)

K
∑

ŝI (ω) ≈Fig. 1. Time-channel domain representation of observed signals for eachfrequency bin.

|X(ω)| ≈ X S (ω) + X I (ω) .

X̂ I (ω) ≈ âI (ω) ŝI (ω) ,sIk (ω) .

(9)

k=1

According to the equations above, the observed signal
model in the amplitude domain composed of a target andbackground far noises can be expressed by|X(ω)| ≈ |A(ω)| |S(ω)| ,[]|A(ω)| ≈ aS (ω) âI (ω) ,
[ S
]
s (ω)
|S(ω)| ≈.
ŝI (ω)

(10)
(11)
(12)

We conduct background noise suppression on the basis of theabove observation. In particular, we assume that the valueSof aS
1 (ω) is the highest among aj (ω) (j = 1, · · · , M )by placing microphone 1 of the microphone array closest
to the target source, utilizing the flexibility of asynchronousrecording. This assumption is necessary because this method
distinguishes between the target and the noise by considering
the difference in the transfer-function gain among the channels.
In the following, all the modeling and processing can be
carried out in each frequency bin. Therefore, we omit ω forsimplicity.
III. FAR NOISE SUPPRESSION WITH NMF
A. Noise suppression using transfer-function-gain NMF
In this section, we describe noise suppression using theabove observed signal model. This noise suppression employs
transfer-function-gain NMF in the time-channel domain toestimate the parameters of the model. In this method, thetypical decomposition of NMF in audio and acoustic signal
processing [12]–[14], such as decomposition into spectral
patterns and activations, is not used. The parameterization of
the NMF is shown in Fig. 1.
NMF approximates a non-negative matrix as two low-rank
non-negative matrices as follows [9]:|X| ≈ X̃ = ÃS̃.

(13)

The tilde represents the matrices or elements estimated by
NMF. In such a low-rank approximation, the solutions willbe sparse owing to the non-negative constraint. Therefore, thetransfer function gain Ã and the source activation S̃ are identified along with the estimation of the source amplitudes |A|and |S|. Furthermore, NMF minimizes the distance between|X| and ÃS̃. In this method, I-divergence is employed as
the distance regulation. Each parameter is estimated using thefollowing multiplicative update rules:

2379

∑

|Xmn | s̃inãSs̃S + ãI s̃Im∑m i m mãim ← ãimn s̃n∑|Xmn | ãinn S S
ãm s̃m + ãIm s̃Im
∑ is̃im ← s̃imn ãnn(i = S, I),(14)(i = S, I).

(15)

2017 25th European Signal Processing Conference (EUSIPCO)

Moreover, the initial values of ãS are set as (16){
1 − (1 − M )α (m = 1)
S
ãm =
,
α
(otherwise)

Table IE XPERIMENTAL CONDITIONS(16)
observed
signals

where α is an arbitrary number that satisfies 0 < α < 1/(M −1). This is because it is assumed that the value of aS
1 (ω) isthe highest among aS
(ω)
(j
=
1,
·
·
·
,
M
).
Also
the
initialjvalues of ãI are given byãIm =

1(m = 1, · · · , M ),M

simulation [7]convolutive mixture of RIR [17]and clean speech

this experimentspeech recordedwith IC recorders

0.3 s

0.64 s——–35.3–
35.8 dB(A)sampling frequency mismatchamong channels
by artificial
resampling

difference in recording start times
and sampling frequency mismatchamong IC
recorderswithin3 Hz

within0.1 Hzreverberation
timebackgroundnoise except
interference(17)
cause ofphase drift

Then, the enhanced signal Ỹn is obtained from the observed
signal of microphone 1 (X1n ) and a Wiener mask as( S S )2ã1 s̃nỸn = X1n ((18))2 ( I )2 .
SSã1 s̃n + ã1 s̃In

sampling frequency
mismatch

B. Semi-supervised transfer-function-gain NMF

7

If the numbers of microphones and sources are similar, highperformance cannot be expected for the parameter estimation
described in the previous section because the estimation accuracy of NMF is degraded. Some methods have been proposed
to solve this problem including semi-supervised transferfunction-gain NMF [7]. In this method, semi-supervised NMF
in the time-frequency domain [15] is applied to transferfunction-gain NMF. In particular, the transfer function gain
vector of the background noises ãI is obtained by training
using NMF. The training requires the observation of the singlesource duration of the background noises. After that, thetransfer function gain vector of the target a˜S and the activation
vectors s̃S and s̃I are obtained by updating (14) and (15) withãI fixed.
Note that supervised transfer-function-gain NMF [16] is also
an effective method, which requires the training of ãS andãI . Although the performance of supervised NMF is higher
than that of semi-supervised NMF, it is difficult to obtainthe single-source durations of the target in a constantly noisy
environment. Therefore, semi-supervised NMF is employed inthis paper.

350cm

8

200cm
1

interference (loudspeaker)
microphones(IC recorders)

4 200cm

5

3

250cm

Fig. 2. Arrangement of target, interference sources and microphones.
target (loudspeaker)
9 10

8

50cm

7

6

1
2

30cm

A. Experimental conditions

ISBN 978-0-9928626-7-1 © EURASIP 2017

300cm
2

350cm

IV. EXPERIMENTAL EVALUATIONIn this experiment, we confirm the effectiveness of themixing model and transfer-function-gain NMF in a real environment. Table I shows the differences between the conditionsin the simulation [7] and this experiment. The other conditions
are almost equivalent to those in the simulation [7]. Figure 2
shows the arrangement of the target source, the interference
sources and the microphones, and Fig. 3 shows the microphonearrangement. As shown in Fig. 2, the distances between the
microphones and the interference sources vary between 2m and 3.5 m. The microphone placed closest to the target
source is microphone 1. The signal-to-distortion ratio (SDR)
and signal-to-interference ratio (SIR) [18] are used as theevaluation scores. We calculated the evaluation scores of
the unprocessed observation (unproc) and of the observation
processed by the method employing semi-supervised transferfunction-gain NMF with 3 channels, 6 channels and 9 channels(3 ch, 6 ch, 9 ch) as shown in Fig. 3.

target (loudspeaker)

300cm

6

250cm

3ch: 1, 4, 8

6ch: 1, 2, 4, 6, 8, 1059ch: all microphones
except 7

4 3
Fig. 3. Arrangement of microphones (IC recorders).

B. Experimental results
Figure 4 shows the experimental results. In transferfunction-gain NMF employing the observed signal model, theSDRs and SIRs are higher than those of the unprocessed
observation. From the results for unproc and 3 ch, we confirm
that this method can supress the noises by assuming that theobserved signal is composed of a target and a mixed noise andthat it can avoid underdetermination. Furthermore, according
to the results for 3 ch, 6 ch and 9 ch, we confirm that the2380

2017 25th European Signal Processing Conference (EUSIPCO)

-1

4

9

-2

3

8

-3

2

7

-4

1

6

-5

0

5

-6

-1

4

-5

0.55

9ch

0

Estimated transfer-function gain

6ch

5

SIR [dB]

Input SNR [dB]
2

7

12

0

5

10

-2

3

8

-4

1

6

-6

-1
-5

Input SNR [dB]noise2

0.4

noise3

0.35

noise4noise5

0.3

noise6

0.25

noise7noise8ch4

ch8

Estimation by 3 channels
0.3
5Fig. 4. Experimental results.

greater the number of microphones used to suppress noises,
the greater the performance of noise suppression, similarly toin the simulation [7].
The tendency of the results in this experiment and thesimulation is similar. However, the improvements in the SDR
and SIR in this experiment are less than those in the simulation[7]. This is because the distance between the microphones andthe interference sources in this experiment is shorter than thatin the simulation. Also, according to Table I, the conditions,such as the reverberation time, the background noise andthe sampling frequency mismatch, in this experiment are lessfavorable than those of the simulation. Thus, we can conclude
that the background noises are effectively suppressed in a realenvironment by the proposed method.
C. Estimation of Ã
We investigate the estimation of the transfer-function gainÃ (ãS and ãI ) to confirm the effectiveness of processing
multiple noises as a single basis. We employ the transferfunction-gain NMF to a single sound comprising all sources(1 target and 8 noises), assuming that we know the numbersof sources and transfer-function gain bases we need. Note thatthe sensitivity is almost equal among all the microphones, andthat the values do not express the true observation because
they are normalized in the range of each source. Figure 5
shows the observation of Ã with 3, 6 and 9 channels. The
transfer-function gain of the target has a greater difference
among the channels than that of the noises, and all the transferfunction gains of the noises are similar among the channels.
The entry “ch1” is the transfer-function gain of microphone 1,
which is placed closest to the target. Then “ch1” is by far thehighest transfer-function gain of the target, but regarding thenoises, an exceptionally high value is not found. Therefore,
we consider that the assumptions given by (10) – (12) clearly
hold. Furthermore, the dispersion of the transfer-function gainof the noises is different between “Estimation by 3 channels”
and “Estimation by (6 or 9) channels” in Fig. 5. This is because
the minimum distance between the IC recorders is different (3channels: 52[cm], 6 and 9 channels: 30[cm]). It is clear thatthe longer the distances among the microphones, the longer
the distances necessary between the microphone array and thenoise source.

ISBN 978-0-9928626-7-1 © EURASIP 2017noise1

0.45

ch1

4
0

target0.5

0.2

Estimated transfer-function gain

3ch

target0.25

noise1noise2

0.2

noise3noise4

0.15

noise5noise6

0.1

noise7noise8

0.05
ch1

ch2

ch4

ch6

ch8

ch10

Estimation by 6 channels
0.25
Estimated transfer-function gain

SDR [dB]

unproctargetnoise1

0.2

noise2noise3
0.15

noise4noise5noise6

0.1

noise7noise8

0.05
ch1

ch2

ch3

ch4

ch6

ch7

ch8

ch9 ch10

Estimation by 9 channels

Fig. 5. Estimated transfer function gains (the components of Ã(ãS , ãI )) in
3ch, 6ch and 9ch case from top to bottom.

V. CONCLUSION
In this paper, we examined the performance of multiple far
noise suppression using transfer-function-gain NMF by recording a target and far noises with asynchronous microphones in areal environment. We confirmed that this method can suppress
background far noises effectively in a real environment, regardless of the diversity of distances between the microphones
and interference sources.
ACKNOWLEDGEMENT
This work was partially supported by the Japan Society for
the Promotion of Science (JSPS) KAKENHI through Grantin-Aid for Scientific Research under Grant 16H01735.

2381

2017 25th European Signal Processing Conference (EUSIPCO)R EFERENCES[1] J. L. Flanagan, D. A. Berkley, G. W. Elko, and M. M. Sondhi,“Autodirective microphone systems,” Acustica, pp. 58–71, Feb. 1991.
[2] Y. Kaneda and J. Ohga, “Adaptive microphone-array system for noise
reduction,” IEEE Trans. Acoust. Speech Signal Process., pp. 1391–1400,
Dec. 1986.
[3] D. H. Dugeon and D. E. Johnson, Array Signal Processing: Concepts
and Techniques, Prentice Hall, Jan. 1993.
[4] H. L. Van Trees, Ed., Optimum Array Processing, Wiley, May 2002.
[5] E. Robledo-Arnuncio, T. S. Wada, and B.-H. Juang, “On dealing withsampling rate mismatches in blind source separation and acoustic echo
cancellation,” Proc. WASPAA, pp. 34–37, Oct. 2007.
[6] Z. Liu, “Sound source separation with distributed microphone arrays in
the presence of clock synchronization errors,” Proc. IWAENC, pp. 1–4,
Sept. 2008.
[7] Y. Murase, H. Chiba, N. Ono, S. Miyabe, T. Yamada, and S. Makino,“Diffuse noise suppression with asynchronous microphone array basedon amplitude additivity model,” Proc. APSIPA, pp. 599–603, Dec. 2015.
[8] T. Kako, K. Kobayashi, and H. Ohmuro, “Speech enhancement using
amplitude-spectrum beamformer for an asynchronous distributed microphone array,” IEICE Technical Report, pp. 51–56, Jan. 2014.
[9] D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix
factorization,” Proc. NIPS, pp. 556–562, Dec. 2001.
[10] M. Togami, Y. Kawaguchi, H. Kokubo, and Y. Obuchi, “Acoustic echo
suppressor with multichannel semi-blind non-negative matrix factorization,” Proc. APSIPA, pp. 522–525, Dec. 2010.
[11] N. Ito, H. Shimizu, N. Ono, and S. Sagayama, “Diffuse noise suppression using crystal-shaped microphone arrays,” IEEE TASLP, vol. 19,
no. 7, pp. 2101–2110, Sept. 2011.
[12] P. Smaragdis and J. C. Brown, “Non-negative matrix factorization formusic transcription,” Proc. WASPAA, pp. 177–180, Oct. 2003.
[13] T. Virtanen, “Monaural sound source separation by nonnegative matrix
factorization with temporal continuity and sparseness criteria,” IEEETASLP, vol. 15, no. 3, pp. 1066–1074, Mar. 2007.
[14] C. Févotte, N. Bertin, and J.-L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence. with application to music
analysis,” Neural Comput., vol. 21, no. 3, pp. 793–830, Mar. 2009.
[15] P. Smaragdis, B. Raj, and M. Shashanka, “Supervised and semisupervised separation of sounds from single-channel mixtures,” Proc.
ICA, pp. 414–421, Sept. 2007.
[16] H. Chiba, N. Ono, S. Miyabe, Y. Takahashi, T. Yamada, and S. Makino,“Amplitude-based speech enhancement with nonnegative matrix factorization for asynchronous distributed recording,” Proc. IWAENC, pp.
204–208, Sept. 2014.
[17] E. A. Habets,“Room impulse response generator,” Oct. 2008,
Available at www.audiolabs-erlangen.de/fau/professor/habets/software/
rir-generator.
[18] E. Vincent, R. Gribonval, and C. Févotte, “Performance measurementin blind audio source separation,” IEEE TASLP, vol. 14, no. 4, pp.
1462–1469, July 2006.

ISBN 978-0-9928626-7-1 © EURASIP 20172382