Ego-Noise Reduction for Hose-Shaped Rescue Robot Using Basis-Shared Semi-Supervised
Independent Low-Rank Matrix Analysis
Moe Takakusaki1 , Daichi Kitamura2 , Nobutaka Ono3 , Shoji Makino1 ,Takeshi Yamada1 , Hiroshi Saruwatari2
1

University of Tsukuba, 1–1–1 Tennodai, Tsukuba, Ibaraki 305-8577, JapanThe University of Tokyo, 7–3–1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
3
Tokyo Metropolitan University, 6-6 Asahigaoka, Hino, Tokyo 191–0065, Japan2

1. Introduction
It is an important task to develop robots for coping with largescale disasters. Robots working in a disaster site are required inemergency response and in the restoration of the disaster site,which are difficult or dangerous tasks for humans. There aretasks for modern-day robots in disaster response, but their functions are insufficient outdoors and their ability to respond to
unexpected situations is unsatisfactory. For example, a typicalrobot cannot smoothly move in a disaster site and evaluate thesituation, as well as act in an environment that does not fit its
working condition. To support the development of robots in adisaster site, which can overcome some of the problems of conventional robots, the Council for Science, Technology and Innovation promoted the ImPACT Tough Robotics Challenge [1].
In this research and development program, we aim to realize
remote-controlled and autonomous robots that are effective in
extreme situations, and develop technologies that will become
the basis for the development of outdoor robots.
The hose-shaped rescue robot (Fig. 1) [2] was developed forentering narrow and dark places covered with rubble in a disaster site and finding survivors inside it. This robot vibrates its
body by the motors attached at each node to move forward without any control devices or operators. The purpose of the robot
is to detect voice of survivors buried in rubbles using the microphones, which are also attached around the body of the robot.
However, since the vibration causes loud noise (ego-noise), theobserved voice and ego-noise must be separated for robustly detecting the survivors. In [3], a novel multichannel source separation technique called independent low-rank matrix analysis(ILRMA) [4, 5, 6] was applied to reduce ego-noise of the robot,
where ILRMA does not require any prior information, e.g., array geometry of microphones, direction of sources, or sample
sounds of sources. To improve its performance, the utilizationof a sample sound of ego-noise for learning its timbers (spectral patterns) can be considered because the sample sound of
ego-noise can easily be obtained by operating the robot in advance. For this reason, in this paper, we propose a new effective
semi-supervised algorithm based on ILRMA. The experimental analysis shows that the proposed method can outperform theconventional methods of ego-noise reduction.
2. Ego-noise of hose-shaped rescue robot

Vibration motor
Microphone

Figure 1: Hose-shaped rescue robot.
the robot. According to the operation principle of the robot,
very loud ego-noise is always mixed into the microphones. The
main factors for the ego-noise are considered to be the vibration
sound generated by the vibration motors and fricative sound. In
an actual disaster site, the voice of a survivor seeking help is notloud enough to capture, and it is fainter than the ego-noise. To
capture the voice in such a situation, it is necessary to separate
the voice from the recorded noisy sound.
2.2 Conventional method for ego-noise reduction
2.2.1 Ego-noise reduction based on blind source separation
The most popular and reliable approach for audio source separation using microphone array is a blind source separation(BSS) technique [7, 8, 9], which does not require any information about the locations of microphones and sources. The
assumption in BSS is valid for the hose-shape rescue robot because relative positions of the microphones (array geometry)can be changed depending on posture of the robot and the location of an observed voice is also unknown. In [3], a novel
BSS technique, ILRMA [4, 5, 6], is applied to the observed
multichannel signal. ILRMA assumes both statistical independence between sources and low-rank structure of each source intime-frequency domain and separates the sources in a fully blind
manner, which will be introduced in the following sections.
2.2.2 Formulation in BSS
Let both the numbers of sources and microphones be M . The
complex-valued source, observed, and separated signals in eachtime-frequency slot are defined as follows:

sij = (sij,1 , · · · , sij,m , · · · , sij,M )t ,(1)
2.1 Structure of robot and disposition of ego-noiset
xij = (xij,1 , · · · , xij,m , · · · , xij,M ) ,(2)
The hose-shaped rescue robot consists of a hose for the axistand ciliary tape wrapped around it; it moves forward slowly
yij = (yij,1 , · · · , yij,m , · · · , yij,M ) ,(3)against the direction of the cilia by vibrating the ciliary tapewith vibration motors. A camera and a lighting are attached where 1 ≤ i ≤ I (i ∈ N) is the index of frequency bins,to the tip of the robot, in addition to an inertial measurement 1 ≤ j ≤ J (j ∈ N) is the index of time frames, 1 ≤ m ≤unit, microphones, and speakers attached along the length of M (m ∈ N) is the index of sources and channels, and t denotes- 351 -

xij = Ai sij ,Observed signal
Frequency-wisedemixing matrix

Low-rank approximation

Estimated signal
STFT

Update demixing matrix so that estimated signals
are mutually independent and have low-rank
structure in time-frequency domain(4)Frequency
Frequency

the vector or matrix transpose. When the mixing system is timeinvariant and the reverberation time is much shorter than thewindow length used in short-time Fourier transform (STFT), thefollowing approximated mixture becomes valid:

Time

Time

M ×M

where Ai = (ai,1 · · · ai,M ) ∈ C
is a frequency-wise mixFigure 2: Separation mechanism of ILRMA.
ing matrix and ai,m is the steering vector for each source. When
Wi = A−1
can be defined, the separated signal yij is repre- 3. Proposed methodisented as
3.1 Motivation
yij = Wi xij ,(5)
In this paper, similar to [3], we assume that the number of
ego-noise sources is M ′ = M − 1 and only one speech sourceh
M ×M
where Wi = (wi,1 · · · wi,M ) ∈ C
is the frequency-wise(survivor’s voice) is mixed in the observed noisy speech sigdemixing matrix, wi,m is the demixing filter for each source,(mix)(noise), canhand denotes the Hermitian transpose. Note that there exists nal xij . Since the sample sound of ego-noise, xij
easilybeobtainedinadvance,asemi-supervised
approachforscale ambiguity in Wi among the frequency bins. To fix theego-noise
reductioncanbeconsideredto
improve
theseparascale of yij , a back-projection technique [10] is often appliedtion performance. The simplest way to utilize such an egoafter the estimation of Wi .
(noise)is to combine a semi-supervised
The typical BSS algorithms including independent vector noise sample sound xij
NMF
algorithm
[13,
14]
and
ILRMA
as the following steps: (a)analysis (IVA) [8, 9] and ILRMA assume (4) and (5) and es(noise)
is separated by simple ILtimate the frequency-wise demixing matrix Wi . Strictly speak- the ego-noise sample sound xij
ing, the mixing assumption (4), which is called linear time- RMA in advance, (b) the supervised (pre-trained) basis matrix(noise)invariant mixing, is not valid for an observed multichannel sig- for ego-noise, Tm, is obtained from the optimization result′
nal by the hose-shaped rescue robot. This is because the robot of (a), where 1 ≤ m′ ≤ M ′ ∈ N is the index of the ego-noise
always moves very slowly, and relative locations of micro- sources, and (c) we apply another ILRMA to the noisy speechphones and sources are barely changing with time. However, signal x(mix) using the supervised basis matrix T (noise), whereijm′
it is reported that BSS with assumption (4) can achieve high(noise)is fixed and the other variables (activation matrix forTm′
separation performance for a short-time (batch) observed signal
the supervised ego-noise sources, basis and activation matrices[3], which means that the batch-wise algorithm can be used forfor the unknown speech source, and Wi ) are optimized, as wella long-time observation.
as the semi-supervised NMF algorithm [13, 14]. However, this
2.2.3 ILRMA
naive semi-supervised approach may fail to fully receive beneILRMA is a method unifying IVA and nonnegative matrix fits from the sample sound of ego-noise because the scale amcollapse the spectral
factorization (NMF) [11] with Itakura–Saito divergence (IS- biguity in Wi among frequency bins can (noise).
NMF) [12], which allows us to simultaneously model the sta- structures in the supervised basis matrix Tm′
To cope with this problem, in this paper, we propose a newtistical independence between sources and the low-rank sourcewise time-frequency structure. Whereas IVA employs source- semi-supervised algorithm based on ILRMA, which is called
wise frequency vectors, ILRMA estimates sourcewise power basis-shared ILRMA (BS-ILRMA). The overview of the′ pro-′(noise)∈ CM ×M
spectrograms that are approximately modeled by the NMF vari- posed method is depicted in Fig. 3, where Wi(mix)
ables. The demixing matrix Wi and the separated signal yij and Wi∈ CM ×M are the demixing matrices for the egoare optimized so that the spectrogram of each source tends to noise sample signal x(noise) and the observed noisy speech sigij
be a low-rank matrix. This separation mechanism of ILRMA is(mix)(noise)(noise)∈ CI×J
nal xij , respectively, Xm′∈ CI×J and Ym′
I×L
L×J
shown in Fig. 2, where Tm ∈ R≥0 and Vm ∈ R≥0 are the(noise)(noise)′and yij,
basis and activation matrices (NMF variables) for the mth esti- show the spectrograms of m th channel in xij
(mix)
(mix)
I×J
I×J
mated source, respectively, and 1 ≤ l ≤ L (l ∈ N) is the index respectively, Xm∈ C
and Ym∈ Cshow theof the bases. Wi , Tm , and Vm can consistently be estimated in spectrograms of mth channel in x(mix) and y (mix) , respectively,ij
ij
a fully blind manner.
an absolute symbol with dotted exponent for matrices denotes
The cost function in ILRMA is defined as follows:
entry-wise absolute and exponent operations, Tm′ ∈ RI×L
≥0 is
[
]
H
2
∑
∑
|w xij |the shared basis matrix for ego-noise sources, TM ∈ RI×L
≥0 is
∑ i,m
J =
+ log
til,m vlj,m(noise)tv
the unshared basis matrix for the speech source, and Vm′∈l il,m lj,m
i,j,m
l
(mix)
L×J
∑
RL×J
and
V
∈
R
are
the
activation
matrices
for
m
≥0
≥0
− 2J
log | det Wi |,(6)(noise)
(mix)
Y
and
Y
,
respectively.
In this method, we employ′
mimtwo ILRMAs; one is applied to the ego-noise sample soundwhere til,m and vlj,m are the nonnegative elements of Tm and(noise)(noise)(noise)to estimate Wi
and yij, and the other one is
V , respectively. The rank-L matrix T V corresponds to the xijmmm

NMF decomposition and represents an estimated model spectrogram of the mth source. An efficient update algorithm for all
Wi , Tm , and Vm is derived [4, 6] for ILRMA.
For the ego-noise reduction of the hose-shaped rescue robot,
we can assume that ego-noise sources contain similar spectral
patterns with repetitions, and the power spectrogram of egonoise sources tends to be a low-rank matrix. For this reason,
ILRMA is suitable for this task and can effectively capture the
ego-noise components to separate them [3].

(mix)applied to the noisy speech signal xij
(mix)

(mix)

to estimate Wi

and yij . The important point is that the basis matrices for
ego-noise sources, Tm′ , are shared between these two ILRMAs,
and all the variables in these models are simultaneously optimized. Since the shared basis matrices Tm′ must represent the(noise)(mix)
similar spectra in both xij
and xij , the ego-noise spectral patterns will be captured by Tm′ , and the other basis matrix TM will consequently represent spectral patterns of the rest

- 352 -

Spatial model estimation
Demixing
matrixObserved
signal⋮Source model estimation

⋮

⋮

⋮

⋮

⋮

⋮

⋮

⋮

⋮

∑

⋮

(∗)
l αij,m′ l = 1:

1

∑

⋮

(∗)

′
l til,m vlj,m′

≤

(∗)
2
∑ αij,m
′l
(∗)

l

,

(8)

til,m′ vlj,m′where ∗ = {noise, mix}. Also, the second and fifth terms in (7)are concave functions, and we apply the tangent line inequality(∗)
to them using an auxiliary variable cij,m′ ≥ 0 as

Sharedbases

Noise sample
Observed noisy speech⋮that satisfies

Basis Activation
matrix matrix

Powerspectrogram

Separated
signal

⋮

log

∑
l

Figure 3: Overview of BS-ILRMA, where upper and lower
models are simultaneously optimized.

≤(∗)

til,m′ vlj,m′1(∗)
cij,m′

(

∑

)
(∗)
(∗)
til,m′ vlj,m′ − cij,m′(∗)+ log cij,m′ .

(9)lsource, i.e., the target speech. Thus, we expect that the sepa- The equality of (9) and (10) holds if and only if the auxiliary
(mix)
(mix)
rated signals y1, · · · , yM ′ are corresponding to the ego- variables are set as follows:
(mix)
(mix)
(∗)
noise sources in xij , and the other separated signal yMtil,m′ vlj,m′
(∗)
(mix)
,
(10)
α
=
∑
is corresponding to the target speech source in xij . Even if
ij,m′ l(∗)′ til′ ,m′ vl′ j,m′l(noise)(mix)
the scale ambiguities in Wi
and Wiexist, this basis
∑
(∗)
(∗)
til,m′ vlj,m′ .
(11)cij,m′ =sharing indirectly improves the estimation accuracy of spectrallpatterns of ego-noise and speech sources, resulting in more accurate source separation.
We can obtain the majorization function of (7) using (8) and (9)3.2 Cost function and update rules in BS-ILRMA

as

L ≤L+
(noise)We assume that the ego-noise sample sound xij
is ob[()∑(noise) 2 (noise)(noise)∑∑ |yij,m′tained as an M ′ -channel signal and the observed noisy speech′ | αij,m′ l1cl til,m vlj,m′
=
+
(mix)
(noise)(noise)xij
is an M -channel signal, which contains one speechM′til,m′ vlj,m′cij,m′
i,j,m′
lsource. The cost function of BS-ILRMA is defined as the sum()]∑(mix) 2 (mix)(mix)′
of two costs of ILRMA as follows:
1 ∑ |yij,m′ | αij,m′ ll til,m vlj,m′
+
+
,
(mix)
(mix)
 ′
M
′v
t
c
[
]
′
′
il,m
l
ij,m
lj,m
(noise)M∑|yij,m′ |2
1 ∑ ∑(noise)
(12)
L= ′
+
log
til,m′ vlj,m′∑(noise)
M  ′
′v
t
′
il,m
m =1 i,j
l
l
lj,m
c
}
where = denotes equality up to a constant term. The update∑(noise)rule of til,m′ is derived by setting the gradient of L+ to zero and−2J
log | det Wi|
substituting (10) and (11), as
i

]
[
′
(mix)
M
∑
Tm′ ←
|yij,m′ |2
1 ∑ ∑
(mix)
v (
+
+ log
til,m′ vlj,m′∑)( (mix) .2 )
(mix)
u
.2
M ′
′
|
|
|Ym′
u 1 |Ym(noise)(noise) T(mix) Tm =1 i,jl′l til,m vlj,m′
1
V
+
Vm′
u
′
′
(noise) .2(mix) .2
m
M
M
[
]
u
Gm′
Gm′
(mix)
∑
∑
|yij,M |2Tm′ ⊗ t,(mix).−1 (noise) T.−1 (mix) T(noise)
(mix)
1
1
+
+ log
til,M vlj,M
∑
Vm′
+M
Gm′
V m′
(mix)
′ Gm′
M
i,j
l
l til,M vlj,M
(13)
}
∑
(mix)
−2J
log | det Wi| ,(7) where ⊗ and the quotient symbol for matrices denote element(∗)(∗)
iwise multiplication and division, and Gm′ = Tm′ Vm′ .
(noise) H= 4. Experimental evaluation
4.1 Conditions(mix) H (mix)(noise)(mix)
wi,m′ xi,jare the elements of Ym′
and Ym′ ,We compared the ego-noise reduction performance of sim(noise)til,m′ and til,M are the elements of Tm′ and TM , vlj,m′ and ple unsupervised ILRMA, naive semi-supervised ILRMA (SS(mix)(noise)(mix)(noise)ILRMA) described in Sect. 3.1, and proposed semi-supervised
vlj,m are the elements of Vm′
and Vm, and wi,m′(mix)(noise)(mix)
BS-ILRMA using the actual recorded signals by the hoseand wi,m are the row vectors of Wi
and Wi.
shaped rescue robot. The number of microphones used in the(noise)
The update rules of unshared parameters, namely, Wi, experiment was M = 8, namely, we assumed that M ′ =(mix)(noise)(mix)
Wi, TM , Vm′, and Vm
are the same as those in M − 1 = 7 ego-noise sources are mixed in the observed noisy[4, 6]. In this paper, we only derive the update rule of the shared speech signal x(mix) . To produce the ego-noise sample andijbases til,m′ . Since it is difficult to directly minimize (7) w.r.t. the observed mixture signals, the robot was driven in a simutil,m′ , we design a majorization function of (7) and minimize it. lated disaster site, and the speech signal was convolved withSince the first and fourth terms in (7) are convex functions for impulse responses from a survivor to each microphone with
til,m′ , we apply Jensen’s inequality to them for obtaining their SNR = −20 dB. We used the signal-to-distortion ratio (SDR)(∗)
upper bound functions using an auxiliary variable αij,m′ l ≥ 0 [15] as an evaluation measure, which indicates total quality of(noise)where yij,m′

=

wi,m′

(noise)

xi,j

(mix)

and yij,m′- 353 -

Table 1: Experimental conditions

Closed test ( L =10)Open test ( L =10)

Closed test ( L =5)
Open test ( L =5)

ILRMA
SS-ILRMA (open test)
BS-ILRMA (open test)

16 kHz2048 samples
512 samples
L=5,10 ,15

12

Closed test ( L =15)
Open test ( L =15)

14
SDR improvement [dB]

SS-ILRMA (closed test)
BS-ILRMA (closed test)

13
SDR improvement [dB]Sampling frequency
Window length in STFT
Window shift length in STFT
Number of bases in each bases matrix

12

11
10

9
8
7
6

5

10

4

8

10

6
4

20

30

40
50
60
70
Number of iterations

80

90

100

Figure 5: SDR improvements according to methods.

2
0

10

20

30

40
50
60
70
Number of iterations

80

90

100

Figure 4: SDR improvements of different number of bases.
source separation. Table 1 shows the other experimental conditions.
In this experiment, we used two semi-supervised situations
related to a sample sound of ego-noise: one is a closed test using the same ego-noise signal in both the sample sound and theobserved signals, and the other one is an open test using the different ego-noise sample recorded by a different posture of thesame robot. The signal length of the ego-noise sample and observed signals was five seconds.
4.2 Results
4.2.1 Optimal number of NMF bases for BS-ILRMA
We first investigate the optimal number of bases L for the egonoise reduction with BS-ILRMA. Figure 4 shows the result of
BS-ILRMA with various numbers of bases, where the horizontal axis indicates the number of iterations in optimization. We
can confirm that BS-ILRMA with L = 5 achieves the best separation performance in open test. This fact means that when we
increase the number of bases, shared ego-noise basis matrices
have a risk to represent not only the ego-noise components but
also the speech components.
4.2.2 Comparison with conventional and proposed methods
Figure 5 shows the comparison of SDR improvements
achieved by ILRMA, SS-ILRMA, and BS-ILRMA, where thenumber of bases L was set to five for all the methods. From
this result, the proposed method outperforms the other methods
for both closed and open tests. As described in Sect. 3.1, SSILRMA cannot achieve better separation performance after several iterations because the scale ambiguity collapses the structures of supervised bases during the iterative optimization. In
contrast, the separation result of BS-ILRMA becomes higher
along with the iterative optimization.
5. Conclusions
In this paper, we proposed a novel semi-supervised extensionof ILRMA that shares the NMF basis between sample and observed signals. This method is applied to the ego-noise reduction task for the hose-shaped rescue robot. The experimental
results show the efficacy of the proposed method.
Acknowledgment
This work was supported by the Japan Science and Technology
Agency and Impulsing Paradigm Change through Distributive

Technologies Program (ImPACT) designed by the Council for
Science, Technology and Innovation, and partly supported by
SECOM Science and Technology Foundation and JSPS KAKENHI Grant Number JP17H06572.
References[1] Impulsive Paradigm Change through Distributed Technologies Program(ImPACT), http://www.jst.go.jp/impact/program07.html.
[2] H. Namari, K. Wakana, M. Ishikura, M. Konyo and S. Tadokoro, “Tubetype active scope camera with high mobility and practical functionality,”
Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3679–3686, 2012.
[3] M. Takakusaki, D. Kitamura, N. Ono, T. Yamada, S. Makino, and
H. Saruwatari, “Ego-noise reduction for a hose-shaped rescue robot using
determined rank-1 multichannel nonnegative matrix factorization,” Proc.
IWAENC, 2016.
[4] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and H. Saruwatari, “Determined blind source separation unifying independent vector analysis andnonnegative matrix factorization,” IEEE/ACM Trans. ASLP, vol. 23, no. 4,
pp. 654–669, 2015.
[5] D. Kitamura, N. Ono, and H. Saruwatari, “Experimental analysis of optimal window length for independent low-rank matrix analysis,” Proc. EUSIPCO, pp. 1210–1214, 2017.
[6] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and H. Saruwatari, “Determined blind source separation with independent low-rank matrix analysis,” Audio Source Separation, Shoji Makino, Ed. Springer, 31 pages (toappear).
[7] P. Comon, “Independent component analysis, a new concept?,” Signal
Processing, vol. 36, no. 3, pp. 287–314, 1994.
[8] T. Kim, H. T. Attias, S.-Y. Lee, and T.-W. Lee, “Blind source separation exploiting higher-order frequency dependencies,” IEEE Trans. ASLP,
vol. 15, no. 1, pp. 70–79, 2007.
[9] N. Ono, “Stable and fast update rules for independent vector analysis
based on auxiliary function technique,” Proc. WASPAA, pp. 189–192,
2011.
[10] K. Matsuoka and S. Nakashima, “Minimal distortion principle for blindsource separation,” Proc. ICA, pp. 722–727, 2001.
[11] D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” Proc. Advances in Neural Information Processing Systems, vol. 13,
pp. 556–562, 2001.
[12] C. Févotte, N. Bertin, and J.-L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis,” Neural Computation, vol. 21, no. 3, pp. 793–830, 2009.
[13] P. Smaragdis, B. Raj, and M. Shashanka, “Supervised and semi-supervised
separation of sounds from single-channel mixtures,” Proc. ICA, pp. 414–
421, 2007.
[14] D. Kitamura, H. Saruwatari, K. Yagi, K. Shikano, Y. Takahashi, andK. Kondo, “Music signal separation based on supervised nonnegative matrix factorization with orthogonality and maximum-divergence penalties,”
IEICE Trans. Fundamentals, vol. E97-A, no. 5, pp. 1113–1118, 2014.
[15] E. Vincent, R. Gribonval and C. Févotte, “Performance measurementin blind audio source separation,” IEEE Trans. ASLP, vol. 14, no. 4,
pp. 1462–1469, 2006.

- 354 -