Ego Noise Reduction for Hose-Shaped Rescue
Robot Combining Independent Low-Rank Matrix
Analysis and Noise Cancellation
Narumi Mae∗ , Daichi Kitamura† , Masaru Ishimura∗ , Takeshi Yamada∗ , and Shoji Makino∗∗ University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577, Japan

E-mail: {mae, ishimura}@mmlab.cs.tsukuba.ac.jp, takeshi@cs.tsukuba.ac.jp, maki@tara.tsukuba.ac.jp† SOKENDAI (The Graduate University for Advanced Studies), Shonan Village, Hayama, Kanagawa 240-0193, Japan

E-mail: d-kitamura@nii.ac.jp

Abstract—In this paper, we present an ego noise reduction
method for a hose-shaped rescue robot developed for searchand rescue operations in large-scale disasters such as a massive
earthquake. It can enter narrow and dark places covered withrubble in a disaster site and is used to search for disaster victims
by capturing their voices with its microphone array. However,
ego noises, such as vibration or fricative sounds, are mixed
with the voices, and it is difficult to differentiate them froma call for help from a disaster victim. To solve this problem,
we here propose a two-step noise reduction method as follows:(1) the estimation of both speech and ego noise signals froman observed multichannel signal by multichannel nonnegative
matrix factorization (NMF) with the rank-1 spatial constraint,
which was proposed by Kitamura et al., and (2) the application ofnoise cancellation to the estimated speech signal using the noisereference. Our evaluations show that this approach is effective
for suppressing ego noise.

I. I NTRODUCTION
It is important to develop robots for search and rescue
operations in times of large-scale disasters such as earthquakes. Robots are required for emergency responses andfor the restoration of disaster sites, which are difficult anddangerous tasks for humans. The “Tough Robotics Challenge”[1] is one of the research and development programs in the
Impulsing Paradigm Change through Disruptive Technologies
Program (ImPACT), whose aim is to develop five remote andautonomous robots. One of these robots is a hose-shaped
rescue robot [2]. This robot is long and slim like a snakeand makes it possible to investigate narrow spaces into whichconventional remotely operable robots cannot enter. This robot
searches for disaster victims by capturing their voices with amicrophone array attached around itself at regular intervals.
However, there is a serious problem of “ego noise”. Ego noise
is generated by the vibration motors used to move the robot
by the vibrating cilia tape wrapped around the robot. Recently,
many ego noise cancellation methods have been proposed [3]–[6]. This robot has reduced ego noise from the sound recordedby its microphone array compared with those in [7]–[8]. In
addition, the many microphones on the hose-shaped rescue
robot enable the application of the overdetermined source
separation method. However, the microphone arrangementchanges as the robot moves, making it difficult to control

9LEUDWLRQPRWRU
0LFURSKRQH

Fig. 1. Hose-shaped rescue robot.

the microphone array geometry. Therefore, in our previous
work [9], we applied independent vector analysis(IVA) [10]–[12] to the robot. However, IVA cannot capture the specificspectral structures of the sources. Thus, in this study, weapply determined rank-1 multichannel nonnegative matrix
factorization [13]–[14] proposed by Kitamura et al, which canbe interpreted as a method of independent low-rank matrix
analysis (hereafter referred to as ILRMA) on the basis of thefact that the hose-shaped rescue robot has many microphones.
We also apply a time-variant noise canceller to compensate
for the time-invariant assumption of ILRMA. We examine theapplicability of the proposed method for reducing ego noise.
II. H OSE -S HAPED R ESCUE ROBOT AND E GO N OISE
A. Hose-Shaped Rescue Robot
Figure 1 shows an image of the hose-shaped rescue robot
and Fig. 2 shows its structure. The hose-shaped rescue robot
basically consists of a hose as its axis with cilia tape wrappedaround it; it moves forward slowly as a result of the friction
between the cilia and floor through the vibration of the cilia

an overdetermined source separation method. In a determinedsituation, IVA [10]–[12] is a commonly used method.
A. Formulation
We suppose that M sources are observed using M microphones (determined case). The multichannel source and theobserved and separated signals in each time-frequency slot
are as follows:

9LEUDWLRQPRWRUV

ϮϬĐŵ

ϮϬĐŵ
0LFURSKRQHV

Fig. 2. Structure of hose-shaped rescue robot.

sij=(sij,1 · · · sij,M )t ,

(1)

xij

t

= (xij,1 · · · xij,M ) ,

(2)

yij

= (yij,1 · · · yij,M ) ,(3)twhere 1 ≤ i ≤ I and 1 ≤ j ≤ J are indexes of frequency andtime, and t denotes the vector transpose. All the entries ofthese vectors are complex values. When the window size in an
STFT is sufficiently longer than the impulse response between
source and microphone, we can approximately represent theobserved signal asxij = Ai sij .

Fig. 3. Principle of movement of hose-shaped rescue robot [2].

tape induces by the vibration motors. Figure 3 schematicallyshows the principle of movement of the hose-shaped rescue
robot. When the motors vibrate, state (1) changes to state (2)through the friction between the cilia and the floor, then state(2) changes into state (3) as a result the cilia slipping. The
hose-shape rescue robot moves by repeating such changes inthe state. It also performs various sensing functions using sensors such as microphones, cameras, an inertial measurement
unit, and light sensors.
B. Problem in Recording Speech
Recording speech using the hose-shaped rescue robot has aserious problem. during the operation of the robot, very loudego noise is mixed in the input to the microphones. The main
sources of the ego noise are as follows:• Driving sound of the vibration motors,• Fricative sound generated between the cilia and floor,• Noise generated by microphone vibration.
In an actual disaster site, the voice of a person seeking help
may not be sufficiently loud to capture, and it may be smaller
than the ego noise.
III. C ONVENTIONAL M ETHOD
Recently, many ego noise reduction methods have been
proposed [3]–[5]. In [3], noise reduction based on the generalization of K-SVD was proposed, which can be used for an
the underdetermined multichannel situation. Also, the authorsof [4] and [5] have proposed a method of improving of
the performances of ego noise reduction using an adaptive
microphone array geometry. On the other hand, the many
microphones on the rescue robot enable the application of(4)Here, Ai = (ai,1 · · · ai,M ) is an M × M mixing matrix ofhthe observed signals. When Wi = (wi,1 · · · wi,M ) denotes
the demixing matrix, the separated signal yij is representedasyij = Wi xij ,(5)

where h is the Hermitian transpose.
B. Independent Vector Analysis
IVA [10]–[12] is one of the techniques used to solve
the permutation problem [15] and can be applied only in a
determined situation. In this method, we define the sourcecomponent as a vector that consists of all frequency bins, givenasyj,m = (y1j,m · · · yIj,m ).

(6)

IVA can be used to estimate the demixing matrix Wi by
assuming both independence between the sources (vectors) anda higher-order correlation between the frequency bins in eachsource. The cost function in IVA is defined as∑∑1∑
G(yj,m ) −log | det Wi |,(7)
Q(W ) =J jmiwhere J is the number of time frames and G(yj,m ) is acontrast function. When yj,m is a probability density function p(yj,m ), the contrast function G(yj,m ) is given as –log p(yj,m ). In IVA, G(yj,m ) = ||yj,m ||2 is often used[12], where a spherical Laplace distribution is assumed forthe source prior and || · ||2 denotes the L2 norm. For theminimization of (7), fast and stable update rules, which arederived by an auxiliary function technique, have been proposed[16].

Amplitude

Frequency

FrequencyTimenoise estimates obtained by ILRMA as the noise reference
signal.
A. Independent Low-Rank Matrix Analysis
Time

Activation matrix(Time-varying gain)
Amplitude

Observed matrix
Basis matrix(Power spectrogram) (Spectral patterns)

We use ILRMA [13], [14] incorporate a rank-1 spatialmodel in MNMF [22]. Here, we explain the formulation andalgorithm derived by Kitamura et al. MNMF is an extensionof simple NMF for multichannel signals. The observed signals
are represented as
Xij = xij xhij ,Fig. 4. Decomposition model of NMF.

IV. P ROPOSED M ETHOD
IVA requires independence between the sources to estimate the demixing matrix. In general, in IVA, a sphericalmultivariate distribution is assumed as the source model toensure a higher-order correlation between the frequency bins
in all sources. However, this model does not include any
particular information on sources, that is, IVA cannot capture
specific spectral structures of the sources. Thus, the utilizationof nonnegative matrix factorization (NMF) [17]–[19] as thesource model has been proposed, which enables us to capture
the spectral structures.
NMF decomposes a given spectrogram into several spectral
bases T and temporal activations V as shown in Fig. 4, thenthe decomposed components are clustered into each separated
source. Multichannel NMF (MNMF) [20]–[22] is one of thetechniques for clustering the NMF bases and activations using
a sourcewise spatial model. MNMF separately models the
mixing system and the nonnegative power spectra of sources.
However, this method is strongly dependent on its initial values
because there are no constraints in the spatial models.
To solve the problem of MNMF, ILRMA [13], [14] wasproposed, in which a rank-1 spatial model is introduces into
MNMF [22]. This method estimates a demixing matrix while
the represented source is determined using NMF bases, andcan be optimized by the update rules of IVA and conventional
single-channel NMF. Therefore, ILRMA is equivalent to amethod that unifies IVA and NMF.
Since the hose-shaped rescue robot moves very slowly andthe spatial locations of the sources and microphones barelychange, we can assume a linear time-varying mixing system.
In this case, IVA or ILRMA is effective for the separation
because neither requires the locations of the sources and themicrophones. In particular, ILRMA can efficiently capture thetime-frequency structure of the ego noise because it repeatedly
analyzes several types of similar spectra. The demixing filter
in IVA or ILRMA is time-invariant over several seconds. To
achieve time-variant noise reduction, in this study we apply anoise canceller for the post processing of ILRMA to reduce
the remaining time-variant ego noise components. The noise
canceller usually requires a reference microphone to observe
only the noise signal. In this study, we utilize estimates of the(8)

where Xij is the correlation matrix between the channels of
size M × M . The diagonal elements of Xij represent realvalued powers detected by the microphones, and the nondiagonal elements represent the complex-valued correlations
between the microphones. The separation model of MNMF
X̂ij used to approximate Xij is represented as∑∑
Xij ≈ X̂ij =Himtil,m vlj,m ,
(9)
m

l

where m = 1 · · · M is the index of the sound sources. Hi,mis an M × M spatial covariance matrix for each frequency iand source m, and Hi,m = ai,m ahi,m is limited to a rank-1
matrix. til,m ∈ R+ and vlj,m ∈ R+ are the elements of thebasis matrix Tm and activation matrix Vm . In ILRMA, thespatial covariance matrix Hi,m is constrained to be a rank-1
matrix. This rank-1 spatial constraint leads to the following
cost function:[∑ ∑ |yij,m |2∑− 2 log | det Wi |Q=l til,m vlj,m
m
i,j
]
(10)
∑
∑
+
log
til,m vlj,m ,mlnamely, the estimation of Hi,m can be transformed to theestimation of the demixing matrix Wi . This cost function isequivalent to the Itakura–Saito divergence between Xij andX̂ij , and we can derive√−2
Σj |yij,m |2 vlj,m (Σl′ til′ ,m vl′ j,m ), (11)
til,m ← til,m−1
Σj vlj,m (Σl′ til′ ,m vl′ j,m )√−2
Σi |yij,m |2 til,m (Σl′ til′ ,m vl′ j,m ), (12)
vlj,m ← vil,m−1
Σi til,m (Σl′ til′ ,m vl′ j,m )∑rij,m =
til,m vlj,m ,(13)l

Vi,m =1∑ 1
xij xhij ,J j rij,mwi,m ←(Wi Vi,m )

−1

em ,

(14)
(15)

where em is unit vector whose mth element is one. We cansimultaneously estimate both the sourcewise time-frequency
model rij,m and the demixing matrix Wi by iterating (11)–(15) alternately. After the cost function converges, the separated signal yij can be obtained as (5). Note that since theEstimated speech௦   ଵ   ௥ 

Mic.1
Mic.2

+
−

Estimated noise௡ଵ 

Motor.1
Motor.2

Output  ௦ ௥ ௡ଶ 
Estimated noise௥   ் ௡ ௡ଷ ʀʀʀ

ILRMA௡ସ ௡ହ 



Filter :Noisereference௡ ௡଺ 
noisentFig. 5. Noise canceller.

Motor.7

Mic.8 ∶ ଵ 

signal scale of yij cannot be determined as well as by IVA,
we apply a back-projection technique [12] to yij to restore
the scale.
B. Noise Canceller
A noise canceller requires a reference microphone locatednear a noise source. The recorded noise reference signal nr (t)is utilized to reduce the noise in the observed speech signal
s1 (t) as shown in Fig. 5. We here assume that both s1 (t)and nr (t) are simultaneously recorded. The observed signal
contaminated with the noise source can be represented asys (t) = s1 (t) + nr (t).

(16)

We consider that the noise signal nr (t) is strongly correlatedwith the reference noise signal yn (t) and that nr (t) can be
represented by a linear convolution model asnr (t) ≃ n̂r (t) = ĥ(t)t yn (t),(17)

where yn (t) = [yn (t) yn (t − 1) · · · yn (t − N + 1)]t is thereference microphone input from the current time t to thepast N samples, and ĥ(t) = [ĥ1 (t) ĥ2 (t) · · · ĥN (t)]t is theestimated impulse response. From (17), the speech signal s1 (t)is extracted by subtracting the estimated noise ĥ(t)t yn (t) fromthe observation asz(t) = x(t) − ĥ(t)t yn (t),(18)

where z(t) is the estimated speech signal. The filter ĥ(t) canbe obtained by a minimization of the mean square error. In
this paper, we use the normalized least mean square (NLMS)
algorithm [23] to estimate ĥ(t). From the NLMS algorithm,
the update rule of the filter ĥ(t) is given asĥ(t + 1) = ĥ(t) + µz(t)yn (t).
||yn (t)||2(19)

C. Flow of Proposed Method
Figure 6 shows the flow of the proposed method. In Fig. 6,
ys (t) is the speech signal estimated by ILRMA, yn1(t) , . . . ,yn7(t) , are the residual outputs that correspond to the various
components of ego noise, and yn is the sum of such ego
noise components. In the first step, the observed signals are
separated into independent signals via IVA or ILRMA, where
the number of separated signals is the same as the numberof microphones (M = 8). Note that it is not fixed which௡଻ 
Noise canceller: Microphones: Vibration motors

Fig. 6. Flow of proposed method.

channel includes most of the speech components owing toa permutation ambiguity in the outputs of ILRMA and IVA.
Therefore, we must find an estimated signal that includes mostof the speech components which we use as ys (t). In this paper,
we manually choose the estimated signal from the output
signals, although such a signal can be estimated by employing
statistics or spectrograms of the output signals. Since a timeinvariant demixing matrix (demixing filter) is applied for the
separation in the first step, the ego noise, which does notfollow the time-invariant assumption, remains in the separated
speech signal ys (t). In the second step, we apply the noise
canceller with the ego noise reference yn (t). In this step, we
expect that the noise canceller will reduce the residual noise
component in ys (t) because it models the time-variant noise asĥ(t)t yn (t), which can update the filter ĥ(t) at each iteration.
V. E XPERIMENT
A. Conditions
In our experiment, we produced an artificial observed signal
using the hose-shaped rescue robot. This robot consists ofeight microphones and seven vibration motors, and the total length of the robot is approximately 3m. The recorded
speech signal was produced by convoluting a dry speech
signal and the impulse response between a disaster victim
and microphones on the robot. For the noise signal, werecorded actual ego noise by moving the robot in an areathat simulated a disaster site. The observed multichannel
signal was obtained as the sum of these speech and ego
noise signals in each microphone, namely, it was a mixtureof time-invariant speech and time-variant actual ego noise.
In addition, we compared four methods; simple IVA, IVA
with the noise canceller (IVA+NC), simple ILRMA, and the
proposed method (ILRMA+NC), using the signal-to-distortion
ratio (SDR) [24] to evaluate the separation performance. The
other experimental conditions are shown in Table I. Note that
we applied the back-projection method to the channel withthe largest number of speech components for all the methods.
However, we can apply the back-projection method to any of
the channels without losing the advantages of the proposed
method.

TABLE I
E XPERIMENTAL CONDITIONS
Sampling frequency
STFT length
Window shift length
Number of bases
Number of iterations
Filter length of noise canceller
Step size of NLMS
Input SNR

16 kHz1024 samples
STFT length/4
152001600 taps
0.1−5, −10 dB

R EFERENCES



6'5LPSURYHPHQW>G%@VII. ACKNOWLEDGMENTS
This work was supported by the Japan Science and Technology Agency and the Impulsing Paradigm Change through Disruptive Technologies Program (ImPACT) commissioned by theCouncil for Science, Technology and Innovation, and partly
supported by SECOM Science and Technology Foundation.
We would also like to express our gratitude to Pf. Hiroshi
Okuno and Mr. Yoshiaki Bando for providing experimental
data.













LQSXW615>G%@,9$,9$1&,/50$,/50$1&

Fig. 7. SDR improvements for each method.

B. Results
Figure 7 shows the improvement in the SDR for eachmethod. These results it is clearly show that the noise canceller
greatly improves the separation performance for both IVA
and ILRMA. This is because it efficiently reduces the timevariant ego noise components from the estimation result of
IVA or ILRMA. In particular, ILRMA+NC outperforms theother methods for both values of the input SNR cases. ILRMA
achieves the higher ego noise reduction than IVA. This difference is due to the existence of a source model, which is thebases decomposition using NMF, between IVA and ILRMA.
The source model in ILRMA efficiently captures the spectralfeatures of the speech and ego noise signals. However, as the
input SNR increases, the difference between the improvements
for ILRMA and ILRMA+NC becomes small. This resultsfrom the variation of the estimated filter coefficients in thenoise canceller, which is caused by the remaining speechcomponents in yn . The proposed method is more effective
when the input SNR is low, namely, in a noisy environment.
VI. C ONCLUSION
To enhance speech signals recorded by a hose-shaped rescue
robot, we have proposed an ego noise suppression method
using ILRMA and a noise canceller. We evaluated the proposed method by an experimental simulation and compared
IVA, ILRMA, IVA with the noise canceller, and ILRMA with
the noise canceller. It was found that the proposed method
exhibited the performance in therms of the SDR under all
conditions, thus confirmed the efficacy of combining ILRMA
and the noise canceller.

[1] “Impulsive Paradigm Change through Disruptive Technologies Program(ImPACT),” http://www.jst.go.jp/impact/program07.html.
[2] H. Namari, K. Wakana, M. Ishikura, M. Konyo, and S. Tadokoro, “Tubetype active scope camera with high mobility and practical functionality,”
Proc. IEEE/RSJ International Conference on Intelligent Robots andSystems (IROS), pp. 3679–3686, 2012.
[3] A. Deleforge and W. Kellerman, “Phase-optimized K-SVD for signal
extraction from underdetermined multichannel sparse mixtures,” Proc.
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 355–359, 2015.
[4] H. Barfuss and W. Kellerman, “Improving blind source separation
performance by adaptive array geometries for humanoid robots,” Proc.
Workshop on Hands-free Speech Communication and Microphone Arrays (HSCMA), 2014.
[5] H. Barfuss and W. Kellerman, “An adaptive microphone array topology
for target signal extraction with humanoid robots,” Proc. International
Workshop on Acoustic Signal Enhancement (IWAENC), pp. 16–20, 2014.
[6] H. Sawada, J. Even, H. Saruwatari, K. Shikano, and T. Takatani,“Improvement of speech recognition performance for spoken-oriented
robot dialog system using end-fire array,” Proc. IEEE/RSJ International
Conference on Intelligent Robots and Systems(IROS), 2010.
[7] Y. Bando, K. Itoyama, M. Konyo, T. Kazuhiro, K. Yoshii, and H. G.
Okuno, “Human-voice enhancement based on online RPCA for a hoseshaped rescue robot with a microphone array,” IEEE International
Symposium on Safety, Security, and Rescue Robotics (SSRR), 2015.
[8] Y. Bando, K. Itoyama, M. Konyo, S. Tadokoro, K. Nakadai, K. Yoshii,and H. G. Okuno, “Variational bayesian multi-channel NMF forhuman-voice enhancement with a deformable and partially-occluded
microphone array,” Proc. European Signal Processing Conference(EUSIPCO), 2016.
[9] M. Ishimura, S. Makino, T. Yamada, N. Ono, and H. Saruwatari, “Noise
reduction using independent vector analysis and noise cancellation fora hose-shaped rescue robot,” Proc. International Workshop on Acoustic
Signal Enhancement (IWAENC), 2016.
[10] T. Kim, T. Eltoft, and T.-W. Lee, “Independent vector analysis: an extension of ica to multivariate components,” Proc. International Conferenceon Independent Component Analysis and Blind Source Separation, 2006.
[11] A. Hiroe, “Solution of permutation problem in frequency domain ica
using multivariate probability density functions,” Proc. International
Conference on Independent Component Analysis and Blind Source
Separation, 2006.
[12] T. Kim, H. T. Attias, S.-Y. Lee, and T.-W. Lee, “Blind source separation
exploiting higher-order frequency dependencies,” IEEE Trans. Speech
and Audio Processing, vol. 15, no. 1, 2007.
[13] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and H. Saruwatari,“Efficient multichannel nonnegative matrix factorization exploiting rank1 spatial model,” Proc. IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP), pp. 276–280, 2015.
[14] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and H. Saruwatari, “Determined blind source separation unifying independent vector analysis
and nonnegative matrix factorization,” IEEE/ACM Trans. Audio, Speech,
and Language Processing, vol. 24, no. 10, 2016.
[15] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust and precise
method for solving the permutation problem of frequency-domain blindsource separation,” IEEE Trans. Speech and Audio Processing, vol. 12,
no. 5, 2004.
[16] N. Ono, “Stable and fast update rules for independent vector analysis
based on auxiliary function technique,” Proc. IEEE Workshop on
Applications of Signal Processing to Audio and Acoustics (WASPAA),
pp. 189–192, 2011.

[17] D. D. Lee and H. S. Seung, “Learning the parts of objects by nonnegative
matrix factorization,” Nature, vol. 401, 1999.
[18] D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” Proc. Advances in Neural Information Processing Systems,
vol. 13, pp. 556–562, 2001.
[19] A. Cichocki, R. Zdunek, A. H. Phan, and S. Amari, Nonnegative matrix
and tensor factorizations: applications to exploratory multi-way data
analysis and blind source separation, John Wiley & Sons, New York,
2009.
[20] A. Ozerov and C. Févotte, “Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation,” IEEE Trans.
Audio, Speech, and Language Processing, vol. 18, no. 3, 2010.
[21] H. Kameoka, T. Yoshioka, M. Hamamura, J. Le Roux, and K. Kashino,“Statistical model of speech signals based on composite autoregressive
system with application to blind source separation,” Proc. International
Conference on Latent Variable Analysis and Signal Separation, 2010.
[22] H. Sawada, H. Kameoka, S. Araki, and N. Ueda, “Multichannel
extensions of non-negative matrix factorization with complex-valued
data,” IEEE Trans. Audio, Speech, and Language Processing, vol. 21,
no. 5, pp. 971–982, 2013.
[23] E. Hänsler and G. Schmidt, Acoustic Echo and Noise Control: A
Practical Approach, John Wiley&Sons, New York, 2004.
[24] E. Vincent, R. Gribonval, and C. Févotte, “Performance measurementin blind audio source separation,” IEEE Trans. Audio, Speech andLanguage Processing, vol. 14, pp. 1462–1469, 2006.