xxx

MULTICHANNEL HEARING-AID SYSTEM BASED ON BASIS-SHARED
SEMI-SUPERVISED INDEPENDENT LOW-RANK MATRIX ANALYSIS
Masakazu Une1
Yuki Kubo2
Norihiro Takamune2
Daichi Kitamura3
Hiroshi Saruwatari2
Shoji Makino1
1
University of Tsukuba, Graduate School of Systems and Information Engineering, Japan
2
The University of Tokyo, Graduate School of Information Science and Technology, Japan
3
National Institute of Technology, Kagawa College, Japan

ABSTRACT

ever, the efficacy of BS-ILRMA for hearing-aid systems
has not yet been determined because it has some disadvantages compared with the ego-noise separation task; theavailable data length is small; the distance from microphones to noise sources is long; there are various interferers (e.g., human voice and footsteps).
Linear time-invariant filters such as ILRMA and BSILRMA cannot separate the target speech under diffuse
noise conditions in principle [4]. To overcome this limitation, rank-constrained spatial covariance matrix (SCM)
estimation [4] has been proposed as an effective method fora situation in which noise arrives from all directions, i.e.,
diffuse noise condition. Basically, this method estimates afull-rank SCM [5], which represents spatial characteristics
of diffuse noise, just as multichannel nonnegative matrix
factorization (NMF) [6, 7] does. Rank-constrained SCM
estimation consists of the following two steps: estimationof spatial parameters using ILRMA as the first step andsuppression of the noise in the direction of the target source
using the estimated spatial parameters as the second step.
Multichannel NMF is sensitive to parameter initialization
and requires the estimation of an enormous number of parameters, leading to a high computational cost. In contrast
to multichannel NMF, the rank-constrained SCM estimation reduces the number of parameters by using the highly
accurate spatial parameters obtained by ILRMA and restores the lost spatial basis for diffuse noise.
We have developed a multichannel hearing-aid system
including binaural ear-attached microphones and smartphone microphones [8]. The additional microphones froma smartphone increase the total number of microphones
as well as provides spatial information apart from theuser’s ears, although these microphones are not synchronized with the ear-attached microphones. We confirmedthat rank-constrained SCM estimation whose parameters
are preestimated by ILRMA achieves a high-quality separation performance for our developed hearing-aid system [8]. If BS-ILRMA is more effective than ILRMA,
rank-constrained SCM estimation is expected to achieve
higher-quality separation performance by introducing BSILRMA to the first step. In this paper, first, we show
the efficacy of BS-ILRMA for practical data recordedby our developed hearing-aid system. Next, we showthat rank-constrained SCM estimation that employs BS-

Independent low-rank matrix analysis (ILRMA) canachieve highest-quality separation performance among
blind source separation methods, but it still fails to separate
the target speech under diffuse noise conditions. To solve
this problem, rank-constrained special covariance matrix(SCM) estimation has been proposed. This method estimates the full-rank SCM of diffuse noise using spatial parameters estimated by ILRMA as the preprocessing andsuppresses the noise in the direction of the target source.
On the other hand, a semi-supervised extension of simple ILRMA, basis-shared ILRMA (BS-ILRMA), has been
proposed. This method employs two ILRMAs for noise
training and separation and achieves more effective separation than simple ILRMA. BS-ILRMA, as well as ILRMA, is a linear separation filter using estimated spatial
parameters. In this paper, we introduce BS-ILRMA to thespatial parameter estimation as the preprocessing of rankconstrained SCM estimation. We evaluate the practical
performance of the proposed method using our developed
hearing-aid system that consists of binaural microphone arrays and the user’s smartphone microphones.
1. Introduction
When we use a binaural hearing-aid system in a noisy environment, target-speech extraction is necessary because
speech is always contaminated by noise. In binauralhearing-aid systems, blind source separation (BSS) [1]and semi-supervised source separation are suitable because these work well without spatial information or partial source information as training data. Among many
BSS methods, independent low-rank matrix analysis (ILRMA) [2] achieves effective and accurate separation. As asemi-supervised extension of simple ILRMA, basis-shared
ILRMA (BS-ILRMA) has been proposed [3]. BS-ILRMA
was originally proposed for a rescue robot and separates
survivor’s voice from the loud noise the robot generates (ego-noise). In this application, training data of thetarget speech cannot be obtained, but ego-noise can easily be acquired in advance. For hearing-aid systems, BSILRMA is applicable because noise information can be acquired as the observed signal before conversation. How-

763

xxx
where Ncmul (µ, Σ) is a multivariate complex Gaussian distribution with a mean vector µ and the covariance matrix Σ. The steering vector ai,n corresponds to the spatial
basis of the nth source, which constructs the rank-1 SCM
as ai,n aH
i,n . The cost function of ILRMA JILRMA is defined as the negative log likelihood function
X X  |yij,n |2
JILRMA =+ log rij,n
rij,n
n i,j
X
− 2J
log |det W i | + const.,(9)

ILRMA as the parameter initialization process can achieve
high-quality separation performance.
2. Formulation and BSS Algorithms
2.1 Formulation
Let us consider separating a multichannel observed signal,
which is obtained by M microphones capturing the signals
arriving from N sources. The source, observed, and separated signals in each time-frequency slot are denoted as
sij = (sij,1 , . . . , sij,n , . . . , sij,N )> ,(1)

xij = (xij,1 , . . . , xij,m , . . . , xij,M )> ,(2)

y ij = (yij,1 , . . . , yij,n , . . . , yij,N )> ,(3)

iwhere the NMF variables til,n and vlj,n and the demixing matrix W i = A−1
are estimated by minimizing (9),iwhich is the maximum likelihood estimation.

where i = 1, . . . , I, j = 1, . . . , J, n = 1, . . . , N , andm = 1, . . . , M indicate the indexes of the frequency bins,
time frames, sources, and microphones, respectively. The
operator ·> indicates transpose. When each source is adirectional source and the window length of short-time
Fourier transform (STFT) is sufficiently larger than thatof the impulse responses from each source to each microphone, the observed signal xij and the mixing matrix Ai = (ai,1 · · · ai,N ) ∈ CM ×N in each frequency bin
have the relation
xij = Ai sij ,2.3 Rank-Constrained SCM Estimation
Linear time-invariant filters such as ILRMA cannot separate the target speech under diffuse noise conditions inprinciple. To solve this problem, rank-constrained SCM
estimation has been proposed [4]. This estimation focuses
on a situation where one directional target source and diffuse noise are mixed. This method consists of two processes. First, we estimate the linear time-invariant filter W i by applying ILRMA to xij .
Next, the residual noise in the direction of the target source is suppressed by estimating the full-rank noise SCM. ILRMA
outputs M separated signals which consist of one “noisecontaminated target speech” component and M − 1 noiseonly components (see [9] for the physical mechanism of
this phenomenon). The rank of SCM calculated fromM − 1 “noise-only” signals is M − 1 [4]. The diffuse noise
SCM should be a full rank (i.e., rank-M ); however, the estimated SCM lacks one rank (one spatial basis) that corresponds to the target source direction. Since the full-ranknoise SCM is required to suppress the noise in the directionof the target source, in rank-constrained SCM estimation,
the noise SCM is modeled as an addition of rank-(M − 1)
SCM (preestimated by ILRMA) and another rank-1 SCM
whose eigenvalue is estimated.
The rank-constrained SCM estimation assumes the observed signal xij to be the sum of the target source image
vector hij = (hij,1 , . . . , hij,M )> and the diffuse noise image vector uij = (uij,1 , . . . , uij,M )> ; i.e.,(4)where ai,n is the steering vector for each source. If thenumber of microphones is equal to that of sources (M =N ) and Ai is not a singular matrix, the separated signal y ij
can be obtained by estimating the demixing matrix W i =A−1= (wi,1 · · · wi,N )H ∈ CN ×M asiy ij = W i xij ,(5)

where the operator ·H denotes the Hermitian transpose.
2.2 ILRMA
In ILRMA [2], the component of the nth source in eachtime-frequency slot is assumed to be generated from a statistical model that follows the univariate complex Gaussian
distribution as
sij,n ∼ Ncuni (0, rij,n ) ,X
rij,n =
til,n vlj,n ,

(6)
(7)

xij = hij + uij .

l2where Ncuni µ, σ is a univariate complex Gaussian distribution with a mean µ and variance σ 2 , til,n ≥ 0 andvlj,n ≥ 0 are NMF variables of the basis matrix T n ∈RI×L and the activation matrix V n ∈ RL×J , respectively,l = 1, . . . , L is an index of the NMF basis, and L is
the number of bases. rij,n corresponds to the nth source
model. Simultaneously, the observed signal xij follows
the multivariate complex Gaussian distribution because of
the reproductive property, i.e.,
!
X
mul
H
xij ∼ Nc0,
rij,n ai,n ai,n ,
(8)

(10)

The source image vector hij is expressed using a vector(h)corresponding to the target source, ai =: ai,nh out of
the spatial bases ai,1 , . . . , ai,N obtained by ILRMA, and(h)the target source image sij as follows:(h) (h)hij = ai sij ,


(h)
(h)
sij ∼ Ncuni 0, rij ,

(11)
(12)

where nh indicates the index corresponding to the tar(h)get source and rij =: rij,nh is the variance of the tar(h)get source (power spectrogram), where rij is assumed to

n

764

xxx
Spatial model estimation

have sparsity in the time-frequency domain by modelingits prior distribution with the inverse gamma distributionas!
β α  (h) −α−1β(h)rp(rij ; α, β) =exp − (h) , (13)Γ(α) ij
rij

Demixing
matrix<latexit sha1_base64="2V6FHCeIQ6+zreqHk8v4IdKSAOE=">AAAC5nichVE7b9RAEP7i8AjhkUtokCg4cQkKzWmPRLyqCBrKvC4XKXec7M0kWcX2Wus9k2BdSUNBh1LQABIFSgu/gIY/EIn8BEQZJBoKxntWEIqAseyd/Wa+8TczQRKq1ApxOOQNnzp95uzIudHzFy5eGquMT6ykumckNaUOtVkN/JRCFVPTKhvSamLIj4KQWsH2wyLeysikSsfLdjehTuRvxmpDSd8y1K1ca2ckrTZ5q/+4bWnHmiifjrVK6Wa/mzf63UpN1IWz6kmnUTo1lDavK1/Qxjo0JHqIQIhh2Q/hI+VnDQ0IJIx1kDNm2FMuTuhjlLk9ziLO8Bnd5u8m39ZKNOZ7UTN1bMl/Cfk1zKxiShyI9+JIfBb74qv4+ddauatRaNnlMxhwKemOPb+y9OO/rIhPi63frH9qttjAXadVsfbEIUUXcsDPnu4dLd1fnMpviLfiG+t/Iw7FJ+4gzr7Ldwu0+MpVN45DeOI6jpyGmGeccyxjXHLVwRSK6XYcXnSmseP8wM1p/Vh7FZOoceYkM3jmbsX3Crt9vNCTzsqtemOmPrswW5t7UC57BFdxHdNc6Q7m8AjzaLKWZ9jHB3z0trwX3ktvb5DqDZWcy/jDvNe/AP2zq1c=</latexit><latexit sha1_base64="0eS7kwl1cTM/KHQstzbK2uM3Nwg=">AAACxnichVG7SgNRED1Z3++ojWATjIpVuFHxVYk2VmISEwUNYXe9iYv7Yh/xEQRrbRULKwUL8TNs/AHBfIJYRrCxcPZmUSSos+zeuWfmzJ6ZUWxdcz3GqhGpqbmlta29o7Oru6e3L9o/kHMt31F5VrV0y9lUZJfrmsmznubpfNN2uGwoOt9Q9paD+EaZO65mmeveoc3zhlwytaKmyh5BqdVCNM4STFis0UmGThyhrVnRZ2xjBxZU+DDAYcIjX4cMl54tJMFgE5ZHhTCHPE3EOY7RSVyfsjhlyITu0bdEt60QNeke1HQFW6W/6PQ6xIxhjD2xO1Zjj+yevbCPX2tVRI1AyyGdSp3L7ULf6VDm/V+WQaeH3W/Wn5o9FDEntGqk3RZI0IVa55ePLmuZhfRYZZzdsFfSf82q7IE6MMtv6m2Kp69EdUdwOPZFx4bQYNKMKxQrE65S1foUgunmBR50ZuFA+IqY086X9hhGEafMUWLQzMWK5wOb+Vpoo5ObTCSnEtOp6fjiUrjsdgxjBBNUaRaLWMEasqSF4wznuJBWJFPypf16qhQJOYP4YdLJJw1fnb0=</latexit>(noise)Y N0<latexit sha1_base64="sGtEJEoQ0t22P3Y1JypOC8r3akk=">AAAC53ichVE7b9RAEP7i8AjhkSM0SDQnLoHQnPYg4lVF0FChPLgkKHc52ZtJsorttdZ7JsG6loKGDpCogkSBSBv+AA1/AIn8BEQZJBoKxntWUBQBY9k7+818429mgiRUqRVib8AbPHb8xMmhU8Onz5w9N1I5Pzqf6q6R1JQ61GYx8FMKVUxNq2xIi4khPwpCWgg27hfxhYxMqnT8yG4l1I78tVitKulbhjqVaisjabXJH/eWW5Y2rYnyiVirlK71OvnDq71OpSbqwln1qNMonRpKm9aVr2hhBRoSXUQgxLDsh/CR8rOEBgQSxtrIGTPsKRcn9DDM3C5nEWf4jG7wd41vSyUa872omTq25L+E/BpmVjEuvoj3Yl98Fh/EN/Hrr7VyV6PQssVn0OdS0hl5fnHu539ZEZ8W639Y/9RssYrbTqti7YlDii5kn589fbU/d3d2PL8i3orvrH9b7IlP3EGc/ZDvZmj2jatuHIfwxHUcOQ0xzzjnWMa45Kr9KRTTbTu86Exj0/mBm9PKgfYqxlDjzDFm8Mzdiu8UdvNgoUed+ev1xo365MxkbepeuewhXMJlTHClW5jCA0yjyVqeYQe7+Ogp74X30nvdT/UGSs4FHDJv+zfXSKun</latexit>N|<latexit sha1_base64="FOXvN1mQ3zph3kr19QY+cFa3KUw=">AAAC7nichVFNTxRBEH0MfiB+sODFhMvGBcHLpheIoieCF06GDxcw7LKZaRrsMDM96ekdwGH+AIlXOBhjNDHRePSqJy/+ARP5CYYjJF48WNM7wRii1mSmq1/Vq3lV5UW+jA1jh11O97nzFy72XOq9fOXqtb5S/8BirNqaizpXvtLLnhsLX4aibqTxxXKkhRt4vljyNh/k8aVE6Fiq8JHZiUQzcDdCuS65awhqlUZ2G4ngRun0cbbaMGLb6CAdDZWMxe2slT4cyXZX0+pY1ipVWJVZK591aoVTQWGzqvQNDaxBgaONAAIhDPk+XMT0rKAGhoiwJlLCNHnSxgUy9BK3TVmCMlxCN+m7QbeVAg3pnteMLZvTX3x6NTHLGGZf2Tt2zL6w9+w7+/nXWqmtkWvZodPrcEXU6tu7sfDjv6yAToMnv1n/1GywjkmrVZL2yCJ5F7zDT54eHC/cnx9Ob7HX7Ij0v2KH7DN1ECYn/M2cmH9uq2vLEdiyHQdWQ0gzTimWEM6pamcK+XSbFs87U9i2vmfntHaqvYwhVChziBg0c7vie7ndOV3oWWdxrFobr07MTVSmpotl92AQNzFKle5iCjOYRZ20PMMHfMQnJ3L2nRfOy06q01VwruMPc97+AqqQrps=</latexit>

Sharedbases<latexit sha1_base64="G3FZgZWPVcGl6B4aovBr0oSnJGw=">AAAC1nichVFNT9RQFD0URARkBt2YuJk4oK6aO9DIx4roxiVfA5PAZNI+HtDQr7RvKtCMO0Liyp0LVhpZEH+GG/8ACfwE4hITNy64fVMxhKi3ad99595ze+69TuS5iSI67zF6++703x24Nzg0fH+kVB59sJKE7VjIugi9MG44diI9N5B15SpPNqJY2r7jyVVn51UeX01lnLhhsKz2Itn07a3A3XSFrRhqlUvrqRQqjLPlTisLnnVa5WrNJG0VMiesacuaYadAfoeqKGw+LJ9hHRsIIdCGD4kAin0PNhJ+1lADIWKsiYyxmD1XxyU6GGRum7MkZ9iM7vB3i29rBRrwPa+ZaLbgv3j8xsysYJxO6YQu6Rt9oQv69ddama6Ra9nj0+lyZdQqvXu09PO/LJ9Phe0/rH9qVtjEtNbqsvZII3kXostP9z9cLs0ujmdP6RN9Z/0f6Zy+cgdB+kMcL8jFI1091hyJN7pjX2sIeMYZx1LGBVftTiGfblPjeWchdrXv6DltXGuvYAxVzhxjBs+cV0zmTG4vrhd621mZMGuTprVgVedeFssewGM8wXOuNIU5vMY86nrrR/iMY6NhvDUOjMNuqtFTcB7ihhnvrwAXH6Rz</latexit>(mix)

T n0 V (mix)n<latexit sha1_base64="UaENbapslqa/JJRD/V+yu0C5vEk=">AAAC5HichVFNTxRBEH0MooAoK1xMDMnGBYOXTa8SPjwRvHDkw11I2GUz0/RCh/lKT++wONmTN8LNGA8mJJh4IN7gJ3DhD5jITzAeMeHCgZreCcYQtSYzXf2qXs2rKid0ZaQZO++yuu/03L3X29d/f+DBw8Hco6FKFDQVF2UeuIFadexIuNIXZS21K1ZDJWzPccWKs/06ja/EQkUy8N/o3VDUPHvTlw3JbU1QPTdSjQXXgUoq7fWqFi2tvGTck63n7Xrit+u5AisyY/nbTilzCshsIch9RxUbCMDRhAcBH5p8FzYietZQAkNIWA0JYYo8aeICbfQTt0lZgjJsQrfpu0m3tQz16Z7WjAyb019cehUx8xhj39gRu2Bn7Cv7wa7+WisxNVItu3Q6Ha4I64N7j5cv/8vy6NTY+s36p2aNBqaNVknaQ4OkXfAOP3778WL51dJY8ox9Zj9J/yE7Z6fUgR//4l8WxdInU10ZjsCO6dgzGnyacUKxmHBOVTtTSKdbM3jaWYCW8R0zp40b7XmMokCZo8SgmZsVz6Q2ebPQ207lRbH0sjixOFGYncuW3YsneIpxqjSFWcxjAWXS8g5HOMaJ1bD2rffWh06q1ZVxhvGHWQfXOIyqrw==</latexit>

(mix)
X1

<latexit sha1_base64="9sky0l9oKYWRy0BvIUbl1hTzYLM=">AAAC5HichVFLSxxBEP4c8zAa42ougRBYXBVzWXoT8ZGTmEty87Wu4K7LTNtrGudFT++4ZthTbuJNQg6BgEIOklvyE7z4B4T4E0KOBnLJwZreQRFJUsNMV39VX81XVU7oykgzdtZldd+6feduz73evvv9DwZyg0MrUdBUXJR54AZq1bEj4UpflLXUrlgNlbA9xxUVZ+tlGq/EQkUy8Jf1Tihqnr3py4bktiaonntSjQXXgUoq7fWqFi2tvGTck62n7Xryul3PFViRGcvfdEqZU0Bm80HuO6rYQACOJjwI+NDku7AR0bOGEhhCwmpICFPkSRMXaKOXuE3KEpRhE7pF3026rWWoT/e0ZmTYnP7i0quImccoO2VH7JydsC/sB/vz11qJqZFq2aHT6XBFWB/YfbT0+78sj06NN1esf2rWaGDaaJWkPTRI2gXv8OO3H86XXiyOJmPskP0k/QfsjB1TB378i39eEIsfTXVlOALbpmPPaPBpxgnFYsI5Ve1MIZ1uzeBpZwFaxnfMnDYutecxggJljhCDZm5WPJPa5OVCbzorz4ql58WJhYnC7Fy27B48xjDGqdIUZvEK8yiTlnc4wld8sxrWnrVvve+kWl0Z5yGumfXpAuKkqos=</latexit>

W1

⋮

⋮ L⋮

(noise) .2|Y N 0(mix)

WI<latexit sha1_base64="fOZYEAdV9IhL9zf5pUIoWoBgkDU=">AAAC5HichVFNTxRBEH0MIJ/KohcTQ7JxweBl0ysEwRPBi0c+XMCw62am6cUO85We3mFxsiduxJshHExIMPFAuOFP8MIfMJGfQDhCwsWDNb0TjCFqTWa6+lW9mldVTujKSDN21mF1dnXf6ent6x8YvHtvKDd8fzkKGoqLMg/cQK06diRc6YuyltoVq6EStue4YsXZfJnGV2KhIhn4r/V2KKqeveHLuuS2JqiWG6nEgutAJW9abytaNLXyknFPNp+2akmpVcsVWJEZy992SplTQGbzQe4HKlhHAI4GPAj40OS7sBHRs4YSGELCqkgIU+RJExdooZ+4DcoSlGETuknfDbqtZahP97RmZNic/uLSq4iZxxj7zo7YJTtlx+yc/fxrrcTUSLVs0+m0uSKsDe0+XLr+L8ujU+Pdb9Y/NWvUMW20StIeGiTtgrf58fv9y6UXi2PJE/aZXZD+Q3bGvlEHfnzFvyyIxU+mujIcgS3TsWc0+DTjhGIx4ZyqtqeQTrdq8LSzAE3jO2ZO6zfa8xhFgTJHiUEzNyueSW3qZqG3neVnxdJEcXJhsjA7ly27F4/wGONU6Tlm8QrzKJOWHRzhBF+tuvXB+mjttVOtjozzAH+YdfALrkiqdQ==</latexit>

(mix)
Y1

⋮

⋮

⋮

X M⋮0

(mix)

(mix)
<latexit sha1_base64="E9La2cc2eP5CLKa8hH5iJeBEb9A=">AAAC5XichVFNTxRBEH2MqIAoi15ISMjGBcTLpleIiieCF0+GD5ePsMtmprdZO8xXenqHxcnePJlwI3rwYDTxoB71H3jxD5DATyAeMfHiwZreCYQQoCYzXf2qXs2rKid0ZaQZO+iyrnRfvXa9p7fvRv/NWwO5wdtLUdBUXJR54AZqxbEj4UpflLXUrlgJlbA9xxXLzubTNL4cCxXJwH+ht0NR9eyGLzcktzVBtdxIJRZcBypZba9XtGhp5SUTnmzdb9eS5/fatVyBFZmx/FmnlDkFZDYX5PZRQR0BOJrwIOBDk+/CRkTPGkpgCAmrIiFMkSdNXKCNPuI2KUtQhk3oJn0bdFvLUJ/uac3IsDn9xaVXETOPMbbHvrAj9ot9Y4fs37m1ElMj1bJNp9PhirA28GZo8e+lLI9OjZcnrAs1a2zgsdEqSXtokLQL3uHHr94dLT5ZGEvG2Sf2m/R/ZAfsJ3Xgx3/453mx8N5UV4YjsGU69owGn2acUCwmnFPVzhTS6VYNnnYWoGV8x8ypfqw9j1EUKHOUGDRzs+Lp1B4eL/Sss/SgWJosTs1PFWZms2X3YBh3MUGVHmEGzzCHMml5ja/4jh9Ww9qxdq23nVSrK+PcwSmzPvwHghmqww==</latexit>

(mix)

Y N0

(mix)

XM
<latexit sha1_base64="gzDmxJiXWNiQch08PzT9IWLD9Cc=">AAAC5HichVFNTxRBEH0MoIgoi1xMDMmGBYOXTa8SRE8EL1xI+HBhE3bdzDS92GG+0tM7Lkz25I1wI8YDCQkkHAg3/Qlc/AMm8hOMR0y4cLCmd4IxRK3JTFe/qlfzqsoJXRlpxs67rO6e3lu3++703x24d38wN/RgJQqaiosyD9xAVRw7Eq70RVlL7YpKqITtOa5YdTZfpfHVWKhIBv5rvRWKmmdv+LIhua0JqudGqrHgOlBJpf2mqkVLKy+Z8GTrSbuezLfruQIrMmP5m04pcwrIbCHIfUMV6wjA0YQHAR+afBc2InrWUAJDSFgNCWGKPGniAm30E7dJWYIybEI36btBt7UM9eme1owMm9NfXHoVMfMYZ1/ZCbtgX9gp+86u/lorMTVSLVt0Oh2uCOuDOw+XL//L8ujUePub9U/NGg1MG62StIcGSbvgHX68/fFi+eXSePKYHbEfpP+QnbMz6sCPf/LjRbG0b6orwxF4Zzr2jAafZpxQLCacU9XOFNLp1gyedhagZXzHzGn9WnseYyhQ5hgxaOZmxS9Sm7pe6E1n5Wmx9Kw4uThZmJnNlt2HRxjFBFV6jhnMYQFl0vIeJ/iEz1bD2rX2rA+dVKsr4wzjD7MOfgHupqqQ</latexit><latexit sha1_base64="OIRrp/EJ0zGH3KXBShhufcucMD0=">AAAC5HichVFNTxRBEH2MiogKi1xMjMmGBYOXTa8QPjwRvXAyfC1g2GUz09sLHeYrPb3D4mRP3ow3QjiYmGjigXiTn+CFP2AiP8F4hISLB2t6JxhCwJrMdPWrejWvqpzQlZFm7LjLunHzVvftnju9d+/d7+vPDTxYjoKm4qLMAzdQq44dCVf6oqyldsVqqITtOa5YcbZepvGVWKhIBv6S3glF1bM3fNmQ3NYE1XKPK7HgOlDJ6/Z6RYuWVl4y6snW03YtedWu5QqsyIzlLzulzCkgs7kg9xMV1BGAowkPAj40+S5sRPSsoQSGkLAqEsIUedLEBdroJW6TsgRl2IRu0XeDbmsZ6tM9rRkZNqe/uPQqYuYxwn6wA3bCjthX9ov9ubJWYmqkWnbodDpcEdb63z1cPPsvy6NTY/Mf61rNGg1MGa2StIcGSbvgHX78Zv9k8fnCSPKEfWa/Sf8ndsy+Uwd+fMq/zIuFD6a6MhyBbdOxZzT4NOOEYjHhnKp2ppBOt2rwtLMALeM7Zk71c+15DKNAmcPEoJmbFU+nNnG+0MvO8rNiaaw4Pj9emHmRLbsHjzCEUao0iRnMYg5l0vIWB/iGQ6thvbd2rb1OqtWVcQZxwayPfwHzf6qS</latexit><latexit sha1_base64="vaXRcQuGvqpYucdhkav40e5rMro=">AAAC03ichVG7SsRQED3G93vVRrDZdX2BsNxV8QXCoo2V+FoVzCpJvGowL5Ls6hpsxEqwVrBSsBA/w8YfEPQTxFLBxsLJ3aCIqBOSO/fMnMmZGdUxdM9n7LFMKq+orKquqa2rb2hsao61tC55dt7VeFazDdtdURWPG7rFs77uG3zFcbliqgZfVnemwvhygbuebluLftHhOVPZsvRNXVN8gtZm5MSEnJjplRP9ciK9HkuyFBMW/+mkIyeJyGbt2ANkbMCGhjxMcFjwyTegwKNnFWkwOITlEBDmkqeLOMcB6oibpyxOGQqhO/TdottqhFp0D2t6gq3RXwx6XWLG0c3u2TV7YXfshj2x919rBaJGqKVIp1ricme9+bh94e1flkmnj+0v1p+afWxiVGjVSbsjkLALrcQv7J++LIzPdwc97JI9k/4L9shuqQOr8KpdzfH5c1HdFRyOXdGxKTRYNOOAYgXCNapamkI43ZzAw85s7AlfFXPa+NQeRxeSlNlFDJq5WPFYaMOfC/3pLA2k0oOpobmhZGYyWnYNOtCJPqo0ggymMYssaXFxhgtcSlkpkA6lo1KqVBZx2vDNpJMPOy2hQQ==</latexit>

YN

N = N 0 +1

(mix) .2

|Y 1
<latexit sha1_base64="uZcfOerzFdggjL6hifzDRdJHxLw=">AAAC63ichVFNTxRBEH0MfgCKrHgh8bJxF4KXTS8SFU8ELx75WsCwy2amt4EO85We3mFhmD/A1YOJeoGEg/HiVa5c+AMm8hOMR0y8eLCmdwIhRK3JTFe/qlfzqsoJXRlpxs56rN4bN2/d7usfuHN38N5Q4f7wUhS0FRc1HriBWnHsSLjSFzUttStWQiVsz3HFsrP1Mosvx0JFMvAX9U4oGp694ct1yW1NULNQ3qvHgutAJa/TtboWHa28ZNyTncdpM6mme2tJZSJtFkqswowVrzvV3Ckht9mg8A11tBCAow0PAj40+S5sRPSsogqGkLAGEsIUedLEBVIMELdNWYIybEK36LtBt9Uc9eme1YwMm9NfXHoVMYsYZV/ZR3bOTtkn9p39/mutxNTItOzQ6XS5ImwO7Y8s/Povy6NTY/OS9U/NGut4brRK0h4aJOuCd/nx7tvzhRfzo8kYO2Q/SP8BO2Mn1IEf/+RHc2L+vamuDEdg23TsGQ0+zTihWEw4p6rdKWTTbRg86yxAx/iOmVPrQnsRZZQos0wMmrlZ8VRmTy8Wet1ZmqhUn1Qm5yZL0zP5svvwEI8wTpWeYRqvMIsaadnHZ3zBseVZb6x31oduqtWTcx7gillHfwB5eq1p</latexit>⋮|(mix) .2

|Y N 0||Y N|<latexit sha1_base64="6Kj/QYlEWotFeSyTcoZ81ElOYFw=">AAAC7HichVFLTxRBEP4YFRARVr2YeNm4LI/LpheJrxPRiyfDw+URdtnMNA10mFd6esfFYf6AXk08EA+aaGI8ccWjF/+AifwEwxETLhyo6Z1gCAFqMtPVX9VX81WVE7oy0oztdVlXrl7r7um93nej/+bAYOHW7bkoaCkuajxwA7Xg2JFwpS9qWmpXLIRK2J7jinln43kWn4+FimTgv9KboWh49povVyW3NUHNQnmrHguuA5Uspst1LdpaecmoJ9tjaTN5OZJuLSeV8bRZKLEKM1Y861Rzp4TcpoLCH9SxggAcLXgQ8KHJd2EjomcJVTCEhDWQEKbIkyYukKKPuC3KEpRhE7pB3zW6LeWoT/esZmTYnP7i0quIWUSZ/Wbf2AH7xb6zv+zo3FqJqZFp2aTT6XBF2Bx8e3f28FKWR6fG+n/WhZo1VvHYaJWkPTRI1gXv8OM3Hw5mn86Uk2H2me2T/k9sj/2kDvz4H/8yLWa2TXVlOAKvTcee0eDTjBOKxYRzqtqZQjbdhsGzzgK0je+YOa2caC9iCCXKHCIGzdys+ElmD08WetaZG69UH1QmpidKk8/yZffiHu5jlCo9wiReYAo10vIOO9jFD8u33lvb1sdOqtWVc+7glFlfjwFPba23</latexit>

⋮ L⋮

⋮

(mix) .2

<latexit sha1_base64="Ayw4es4+6D747Fi7fCxBfbMK/Hk=">AAAC63ichVFNTxRBEH0MKgt+sOiFxMvGXQxeNr24AfFE4OLJ8OEChl02M02DHeYrPb3j4uz8gb16MFEvmnAwXrzq1Qt/gER+gvGIiRcP1vROMIaoNZnp6lf1al5VOaErI83YyZA1fOHipZHC6NjlK1evjRcnrq9HQUdx0eCBG6hNx46EK33R0FK7YjNUwvYcV2w4+0tZfCMWKpKB/0gfhKLl2Xu+3JXc1gS1i5VeMxZcByp5nG43tehq5SXTnuzeSdvJw7S3nVRn0naxzKrMWOm8U8udMnJbDopf0MQOAnB04EHAhybfhY2Ini3UwBAS1kJCmCJPmrhAijHidihLUIZN6D599+i2laM+3bOakWFz+otLryJmCVPsmL1jp+yIvWdf2c+/1kpMjUzLAZ3OgCvC9nh/cu3Hf1kenRpPfrP+qVljF/eMVknaQ4NkXfABP3724nTt/upUcpu9Zd9I/xt2wj5TB378nR+uiNVXproyHIGnpmPPaPBpxgnFYsI5VR1MIZtuy+BZZwG6xnfMnHbOtJdQQZkyK8SgmZsVz2c2e7bQ8876TLV2t1pfqZcXFvNlF3ATtzBNleawgAdYRoO09PEBH/HJ8qzn1kvr9SDVGso5N/CHWYe/AL9frYY=</latexit>

TN M = N<latexit sha1_base64="00EkY3MnlgI05jxCVjjinxyGyGY=">AAAC03ichVE9TxtBEH2+BEL4spM0kdJYGBCVtU5QPqhQ0qSKsMGABMa6WxY4+b60tz4wJzeIKlLqRKKyJYqIn0HDH0CCn4AoQUqTgrn1CYQQyZzudvbNvLk3M1bg2KFi7DxjPHna1/9s4Png0PDIaDb34uVi6DclF1XuO75ctsxQOLYnqspWjlgOpDBdyxFLVuNLEl+KhAxt31tQrUDUXHPTszdsbiqC1lYjwZUv44V2Pf7WrucKrMi05R86pdQpILU5P3eGVazDB0cTLgQ8KPIdmAjpWUEJDAFhNcSESfJsHRdoY5C4TcoSlGES2qDvJt1WUtSje1Iz1GxOf3HolcTMY4Kdst/sip2wI3bB/j5aK9Y1Ei0tOq0eVwT17PfX83/+y3LpVNi6Y/1Ts8IGPmqtNmkPNJJ0wXv8aPfn1fxMZSKeZF12Sfo77JwdUwdedM0Py6JyoKtLzRHY1h27WoNHM44pFhHOqWpvCsl0axpPOvOxo31Lz2n9Vnse4yhQ5jgxaOZ6xZ8Se3+70IfO4tti6V1xujxdmP2cLnsAbzCGKar0AbP4ijlUSYvEL3TQNapGbOwZ+71UI5NyXuGeGT9uAGWIo9E=</latexit><latexit sha1_base64="SPoTmrcfC9mhnzebbAxnGAAGM6E=">AAACyHichVFLLwNRFP6MV9WjxUZi02iJVXNL45VIhI1ICKoloZGZcdWk88rMtFRjYymx1YUViYX4GTb+gISfIJYkNhbO3E6ICM5k5p77nfOd+c45iq1rrsfYY4PU2NTc0hpqC7d3dHZFot09OdcqOSrPqpZuORuK7HJdM3nW0zydb9gOlw1F5+tKcc6Pr5e542qWueZVbJ435IKp7Wqq7BGUWZxe2o7GWZIJi/10UoETR2DLVvQBW9iBBRUlGOAw4ZGvQ4ZLzyZSYLAJy6NKmEOeJuIcRwgTt0RZnDJkQov0LdBtM0BNuvs1XcFW6S86vQ4xYxhk9+yavbA7dsOe2Puvtaqihq+lQqdS53J7O3LSl3n7l2XQ6WHvi/WnZg+7mBBaNdJuC8TvQq3zy4e1l8zU6mB1iF2yZ9J/wR7ZLXVgll/VqxW+ei6qO4LDsS86NoQGk2ZcpViZcJWq1qfgTzcvcL8zCwfCV8Scdj61x5BAnDITxKCZixVP+jb2udCfTm4kmRpNplfS8ZnZYNkh9GMAw1RpHDOYxzKypKWAU5yhJi1ItrQvVeqpUkPA6cU3k44/AKKfnls=</latexit>

Figure 1. Overview of BS-ILRMA, where upper andlower models are simultaneously optimized.
the application to hearing-aid systems, the observed unvoiced period, which does not contain speech and conversation, can be obtained as the noise samples (training signal of diffuse noise) in advance. Therefore, BSILRMA is applicable for the hearing-aid system. Under
the M = N condition, BS-ILRMA assumes that N 0 =M 0 = N − 1 noise sources and one target source exist. Let(noise)(noise) >(mix)
x(noise)= (x(noise)=
ij 0
ij 0 ,1 , . . . , xij 0 ,m0 , . . . , xij 0 ,M 0 ) and xij(mix) >
0(x(mix)
,
.
.
.
,
x
)
be
the
preparedMchnoise
samplesij,1ij,Mand M mixture signals, respectively. j 0 = 1, . . . , J 0 and
m0 = 1, . . . , M 0 indicate the indexes of the time frameand sources for noise samples, respectively.

(16)

We can consider a simple semi-supervised ILRMA by
employing pretrained basis matrices for noise sources,
which is a similar approach to the semi-supervised
NMF [10]. We call this method semi-supervised ILRMA(SS-ILRMA) in this paper. SS-ILRMA trains the N 0 noisebases T (noise)∈ RI×Ln0≥0 by applying simple ILRMA to the(noise)noise samples xij 0in advance and separates the M channel observed signals x(mix)by another ILRMA whileij
fixing the trained noise basis matrices T (noise), where then0index n0 = 1, . . . , N 0 indicates the source of the noise
samples. However, this naive semi-supervised approach
fails to fully receive the benefits of employing the noise
sample. This is because the scale ambiguity in W i amongfrequency bins can collapse the spectral structures in the
supervised basis matrix T (noise)
[3].
n0

#
+ const.,(17)(x)(h) (h) (h) H(u) (u)Rij =rij ai ai+ rij Ri .

(noise)

XM0

|

⋮

⋮

<latexit sha1_base64="Nq8JrhmCUUah+tcNW8u0fgSpoyQ=">AAAC5XichVFLTxRBEP4YXwgiq1xMSMyG5XnZ9CpR8UTg4sWE18Im7LqZaZqlw7zS0zsuTvbmyYQbwYMHIwkH8Kj/wIt/wAR+AuGIiRcP1vROMIQoNZnp6q/qq/mqygldGWnGjrusa9dv3LzVfbun907f3f7cvfvLUdBUXJR54Aaq4tiRcKUvylpqV1RCJWzPccWKszmbxldioSIZ+Et6KxQ1z274cl1yWxNUzz2sxoLrQCWV9quqFi2tvGTck62Jdj15Odau5wqsyIzlLzulzCkgs7kgd4Qq1hCAowkPAj40+S5sRPSsogSGkLAaEsIUedLEBdroIW6TsgRl2IRu0rdBt9UM9eme1owMm9NfXHoVMfMYYT/YATtj39lndsJ+/7NWYmqkWrbodDpcEdb73z1Y/HUly6NTY+Mv67+aNdbxzGiVpD00SNoF7/DjN+/PFp8vjCSjbI+dkv5P7Jh9ow78+CffnxcLH0x1ZTgCr03HntHg04wTisWEc6ramUI63ZrB084CtIzvmDmtnWvPYxgFyhwmBs3crHgqtSfnC73sLD8qlh4XJ+cnC9Mz2bK7MYghjFOlp5jGC8yhTFre4hBf8NVqWNvWjrXbSbW6Ms4ALpj18Q99PqrB</latexit>

i,j

(h)

(noise) .2|Y 1<latexit sha1_base64="kgghwTUdciFAPSrl7tQl9/Bpipw=">AAAC7XichVE7b9RAEP5iXiE8ckCDRHPikig0p70Q8aoiaCjz4JKg3OVkbyZhFdtrrfdMguM/gGgRBQgJJIQQHS10NPwBJPITEGWQaCgY71lBKALGsnf2m/nG38wESahSK8TukHfo8JGjx4aPj5w4eer0aO3M2cVU942kttShNsuBn1KoYmpbZUNaTgz5URDSUrB5q4wvZWRSpeM7djuhbuRvxGpdSd8y1KtN7HQyklab/G6x2rG0ZU2UT8ZapXSp6OWtYmc1b04VvVpDNIWz+kGnVTkNVDara1/QwRo0JPqIQIhh2Q/hI+VnBS0IJIx1kTNm2FMuTigwwtw+ZxFn+Ixu8neDbysVGvO9rJk6tuS/hPwaZtYxLj6LN2JPfBJvxVfx86+1clej1LLNZzDgUtIbfXh+4cd/WRGfFvd+s/6p2WId15xWxdoTh5RdyAE/e/Bkb+HG/Hg+IV6Kb6z/hdgVH7mDOPsuX83R/FNX3TgO4b7rOHIaYp5xzrGMcclVB1Mop9t1eNmZxpbzAzentX3tdYyhwZljzOCZuxVfL+3K/kIPOotTzdbl5vTcdGPmZrXsYVzARUxypauYwW3Mos1aHuEd3uODp73H3jPv+SDVG6o45/CHea9/AdPJrk0=</latexit><latexit sha1_base64="rad0DTW0Ob8V9V/8pZwN+zShmAo=">AAAC5nichVE7b9RAEP5iXiE8coEGiYITl6DQnPZCFB5VBA1lXpcE5Y6TvZkkq9hea71nEqwraSjoUAoaQKJAackvSJM/gER+AqIMEg0F4z0rCEXAWPbOfjPf+JuZIAlVaoU4HPBOnT5z9tzg+aELFy9dHq6MXFlMdddIakodarMc+CmFKqamVTak5cSQHwUhLQWbj4r4UkYmVTpesNsJtSN/PVZrSvqWoU7lRisjabXJn/SetixtWRPl47FWKd3udfJGr1OpibpwVj3pNEqnhtJmdOULWliFhkQXEQgxLPshfKT8rKABgYSxNnLGDHvKxQk9DDG3y1nEGT6jm/xd59tKicZ8L2qmji35LyG/hplVjInP4qM4EgdiV3wVP/9aK3c1Ci3bfAZ9LiWd4ZfX5n/8lxXxabHxm/VPzRZruOe0KtaeOKToQvb52fOdo/kHc2P5LfFefGP978Sh2OcO4uy7/DBLc29cdeM4hGeu48hpiHnGOccyxiVX7U+hmG7b4UVnGlvOD9ycVo+1VzGKGmeOMoNn7lZ8v7Cp44WedBYn6o079cnZydr0w3LZg7iOmxjnSncxjceYQZO1vMAuPmHP2/Beea+9nX6qN1ByruIP897+AgKyq1k=</latexit>⋮<latexit sha1_base64="30dFeAEf61oiy+5evuIfQDWXARo=">AAAC5HichVFNTxRBEH2MooAoi15MjMnGBYOXTY8QUE8ELxz5cFkSdtnM9PYuHeYrPb3j4mRP3gg3YzyYkGjigXjTn+DFP2AiP8F4xIQLB2p6JxBC1JrMdPWrejWvqtzIk7Fm7HDAunJ18Nr1oeGRG6M3b40Vxm+vxWFHcVHhoReqddeJhScDUdFSe2I9UsLxXU9U3e3nWbyaCBXLMHihdyJR9512IFuSO5qgRuF+LRFchyqt9jZrWnS18tMpX3Yf9Rqp3WsUSqzMjBUvO3bulJDbUlj4iRqaCMHRgQ+BAJp8Dw5iejZggyEirI6UMEWeNHGBHkaI26EsQRkOodv0bdNtI0cDumc1Y8Pm9BePXkXMIibZD3bAjth39pn9Yid/rZWaGpmWHTrdPldEjbHdu6vH/2X5dGpsnbP+qVmjhSdGqyTtkUGyLnifn7x6d7T6bGUyfcg+st+k/wM7ZN+ogyD5wz8ti5X3proyHIGXpmPfaAhoxinFEsI5Ve1PIZtu3eBZZyG6xnfNnJpn2ouYQIkyJ4hBMzcrfprZ7NlCLztrj8v2dHlmeaY0v5Avewj38ABTVGkO81jEEiqk5TUO8AVfrZa1Z72x3vZTrYGccwcXzNo/BalcqnM=</latexit>

where the operator diag(k1 , k2 , . . .) generates the squarediagonal matrix with the arguments on the main diagonal,(u)
R0 i is the noise SCM estimated by ILRMA whose rank
is M − 1, bi is a unit eigenvector corresponding to the zero(u)eigenvalue of R0 i and λi is the weight variable. Note(u)that, in Ri , only λi is the variable to be optimized be(u)cause R0 i and bi are given by ILRMA as fixed values inadvance. By modeling the prior distribution of the target
source variance in (13), we express the negative log posterior function L of the rank-constrained SCM estimationas"X(h) (u)(x) −1(x)
L(rij , rij , λi ) =xH
xij + log det Rijij (Rij )rij

Y1<latexit sha1_base64="E4rlX32BpaGLp+5ByyMVsNddtvI=">AAAC5HichVFNTxRBEH2MX4Aoi15MjMnGBYOXTY8QUE8ELxz5cGETdtnM9PYuHeYrPb3j4mRP3gg3YzyYkGjigXjTn+DFP2AiP8FwhISLB2p6JxBC1JrMdPWrejWvqtzIk7Fm7GDAunL12vUbg0PDN0du3R4tjN1ZjcOO4qLCQy9UVdeJhScDUdFSe6IaKeH4rifW3K0XWXwtESqWYfBSb0ei7jvtQLYkdzRBjcKDWiK4DlVa7W3UtOhq5aeTvuw+7jVSu9colFiZGSteduzcKSG3xbDwCzU0EYKjAx8CATT5HhzE9KzDBkNEWB0pYYo8aeICPQwTt0NZgjIcQrfo26bbeo4GdM9qxobN6S8evYqYRUywn2yfHbEf7Av7zf78tVZqamRatul0+1wRNUZ37q2c/Jfl06mxec76p2aNFp4arZK0RwbJuuB9fvL6/dHK8+WJ9BH7xA5J/0d2wL5TB0FyzD8vieUPproyHIFXpmPfaAhoxinFEsI5Ve1PIZtu3eBZZyG6xnfNnJpn2osYR4kyx4lBMzcrfpbZzNlCLzurT8r2VHl6abo0N58vexD38RCTVGkWc1jAIiqk5Q328RXfrJa1a7213vVTrYGccxcXzNo7BavSqnQ=</latexit>

(u)

β

⋮

W1

⋮

Ri =R0 i + λi bi bH
(15)
i,
X
1
(u)
H
2
H
2
R0 i =W −1
i diag(|w i,1 xij | , . . . , |w i,nh−1 xij | ,J j(h)+ (α + 1) log rij +<latexit sha1_base64="0AMSupYIsXZzd1JfMbpQDEOVg0U=">AAAC53ichVFNTxRBEH2MX4AfrHoh4bJxQfGy6VUC6onIhSMf7kLCrpuZpoEOM92Tnt4RnOyVAxdvauIJEg9Gr/AHvPgHSOAnGI+YePFgTe8EY4hak5muflWv5lVVEIcysYyd9HkXLl66fKV/YPDqtes3hko3bzUS3TFc1LkOtVkO/ESEUom6lTYUy7ERfhSEYinYnMnjS6kwidTqmd2ORSvy15Vck9y3BLVL5WYquNUma3SfN63YsibKxpWWibjfbWfqXrddqrAqc1Y+79QKp4LC5nTpGE2sQoOjgwgCCpb8ED4SelZQA0NMWAsZYYY86eICXQwSt0NZgjJ8Qjfpu063lQJVdM9rJo7N6S8hvYaYZYyxI/aBnbIv7CP7yn7+tVbmauRatukMelwRt4d2hxd//JcV0Wmx8Zv1T80Wa3jktErSHjsk74L3+OnLN6eLTxbGsrtsn30j/XvshH2mDlT6nb+fFwvvXHXjOAIvXMeR06BoxhnFUsI5Ve1NIZ9uy+F5Zxpbzg/cnFbPtJcxigpljhKDZu5W/Di3ybOFnncaD6q1h9WJ+YnK9NNi2f0YwR2MU6UpTGMWc6iTlh18wgEOPem98l57b3upXl/BuY0/zNv7BRxsq8Q=</latexit>(noise)

X1

Noise sample
Observed mixture

(u)

−1 H
2
H
2
0,|wH
i,nh+1 xij | , . . . , |w i,N xij | )(W i ) ,T n0 V n0<latexit sha1_base64="G3FZgZWPVcGl6B4aovBr0oSnJGw=">AAAC1nichVFNT9RQFD0URARkBt2YuJk4oK6aO9DIx4roxiVfA5PAZNI+HtDQr7RvKtCMO0Liyp0LVhpZEH+GG/8ACfwE4hITNy64fVMxhKi3ad99595ze+69TuS5iSI67zF6++703x24Nzg0fH+kVB59sJKE7VjIugi9MG44diI9N5B15SpPNqJY2r7jyVVn51UeX01lnLhhsKz2Itn07a3A3XSFrRhqlUvrqRQqjLPlTisLnnVa5WrNJG0VMiesacuaYadAfoeqKGw+LJ9hHRsIIdCGD4kAin0PNhJ+1lADIWKsiYyxmD1XxyU6GGRum7MkZ9iM7vB3i29rBRrwPa+ZaLbgv3j8xsysYJxO6YQu6Rt9oQv69ddama6Ra9nj0+lyZdQqvXu09PO/LJ9Phe0/rH9qVtjEtNbqsvZII3kXostP9z9cLs0ujmdP6RN9Z/0f6Zy+cgdB+kMcL8jFI1091hyJN7pjX2sIeMYZx1LGBVftTiGfblPjeWchdrXv6DltXGuvYAxVzhxjBs+cV0zmTG4vrhd621mZMGuTprVgVedeFssewGM8wXOuNIU5vMY86nrrR/iMY6NhvDUOjMNuqtFTcB7ihhnvrwAXH6Rz</latexit>(noise)<latexit sha1_base64="jQtDTSz8jnegw8XMAWZXgtJhd90=">AAAC53ichVE9b9RAEH0xHwkJkAMaJJoTl0BoTnsk4quKoKFBygeXnJQ7TvZmElaxvdZ6zyRY11LQ0AESVZAoELTJH0jDH0AiPwFRBomGgvGeFYQiYCx7Z9/MG7+ZCZJQpVaI/SHv2PETJ4dHTo2OnT5zdrxy7vxSqntGUlPqUJtW4KcUqpiaVtmQWokhPwpCWg427hXx5YxMqnT80G4l1In89VitKelbhrqVajsjabXJW/1HbUub1kT5VKxVStf63fzB1X63UhN14ax61GmUTg2lzenKF7SxCg2JHiIQYlj2Q/hI+VlBAwIJYx3kjBn2lIsT+hhlbo+ziDN8Rjf4u863lRKN+V7UTB1b8l9Cfg0zq5gUn8V7cSA+iQ/iq/j511q5q1Fo2eIzGHAp6Y4/v7j447+siE+Lx79Z/9RssYZbTqti7YlDii7kgJ89fXWweGdhMr8i3opvrH9b7Is97iDOvst387TwxlU3jkN44jqOnIaYZ5xzLGNcctXBFIrpdhxedKax6fzAzWn1UHsVE6hx5gQzeOZuxbcLu3G40KPO0vV6Y7o+Mz9Tm71bLnsEl3AZU1zpJmZxH3NospZn+Igd7HrKe+G99F4PUr2hknMBf5i3/QvSa6ul</latexit>

where rij and Ri are the variance and the full-rank
SCM of diffuse noise, respectively. The SCM of the dif(u)fuse noise Ri is represented by the demixing filter wi,nas(u)Basis Activation
matrixmatrix(noise)WI<latexit sha1_base64="iN+bFr68KJhw4VddL+zsgsr3A9M=">AAAC5nichVE7b9RAEP7i8AjhkUtokCg4cQkKzWmPRLyqCBrKvC45KXec7M0kWcX2Wus9k2BdSUNBh1LQABIFSgu/gIY/EIn8BEQZJBoKxntWEIqAseyd/Wa+8TczQRKq1ApxOOQNnzp95uzIudHzFy5eGquMT6ykumckNaUOtWkFfkqhiqlplQ2plRjyoyCk1WD7YRFfzcikSsfLdjehTuRvxmpDSd8y1K1ca2ckrTZ5q/+4bWnHmiifjrVK6Wa/mzf63UpN1IWz6kmnUTo1lDavK1/Qxjo0JHqIQIhh2Q/hI+VnDQ0IJIx1kDNm2FMuTuhjlLk9ziLO8Bnd5u8m39ZKNOZ7UTN1bMl/Cfk1zKxiShyI9+JIfBb74qv4+ddauatRaNnlMxhwKemOPb+y9OO/rIhPi63frH9qttjAXadVsfbEIUUXcsDPnu4dLd1fnMpviLfiG+t/Iw7FJ+4gzr7Ldwu0+MpVN45DeOI6jpyGmGeccyxjXHLVwRSK6XYcXnSmseP8wM1p/Vh7FZOoceYkM3jmbsX3Crt9vNCTzsqtemOmPrswW5t7UC57BFdxHdNc6Q7m8AjzaLKWZ9jHB3z0trwX3ktvb5DqDZWcy/jDvNe/AAA6q1g=</latexit>

where α > 0, β > 0, and Γ(·) are the shape parameter,scale parameter, and gamma function, respectively.
The diffuse noise uij follows the following multivariate
complex Gaussian distribution and is statistically independent of the target source hij :(u) (u)uij ∼ Ncmul 0, rij Ri
,
(14)
(u)

<latexit sha1_base64="1L9VU7APAmL+MUqCmSPjXMNKGA0=">AAAC5nichVE9b9RAEH0xXyF85AgNEgUnLkGhOe2FiBCqCBro8sHlIuWOk72ZJKvYXmu95yRYV9JQ0KEUNIBEgdLCL6DhDyCRn4Aog0RDwXjPCkIRMJa9s2/mjd/MBEmoUivEwZB34uSp02eGz46cO3/h4mjl0thyqntGUlPqUJuVwE8pVDE1rbIhrSSG/CgIqRVs3S/irYxMqnT8yO4m1In8jVitK+lbhrqVa+2MpNUmb/Ufty3tWBPlk7FWKd3sd/OH/W6lJurCWfW40yidGkqb15UvaGMNGhI9RCDEsOyH8JHys4oGBBLGOsgZM+wpFyf0McLcHmcRZ/iMbvF3g2+rJRrzvaiZOrbkv4T8GmZWMSE+i3fiUHwS++Kr+PnXWrmrUWjZ5TMYcCnpjj67svTjv6yIT4vN36x/arZYxx2nVbH2xCFFF3LAz57sHS7dXZzIb4g34hvrfy0OxEfuIM6+y7cLtPjSVTeOQ9h2HUdOQ8wzzjmWMS656mAKxXQ7Di8609hxfuDmtHakvYpx1DhznBk8c7fi2cJuHy30uLM8VW/cqk8vTNfm7pXLHsZVXMckV5rBHB5gHk3W8hT7eI8P3qb33Hvh7Q1SvaGScxl/mPfqFzcKq28=</latexit>(noise)<latexit sha1_base64="STvE2NMKQuKNCDuWuE2M+VrkDDg=">AAACx3ichVFLLwNRFP46XlWvYiOxadRr1dzSeK2EDRuhFAmNzExvmXRemZmO0ljYii0WViQW4mfY+AMSfoJYkthYOHM7ISI4k5l77nfOd+Y75yi2rrkeY48Rqa6+obEp2hxraW1r74h3dq26VtlReU61dMtZV2SX65rJc57m6XzddrhsKDpfU0qzQXzN546rWeaKt2fzvCFvm1pRU2WPoOzC0FY8yVJMWOKnkw6dJEJbtOIP2EQBFlSUYYDDhEe+DhkuPRtIg8EmLI8qYQ55mohzHCBG3DJlccqQCS3Rd5tuGyFq0j2o6Qq2Sn/R6XWImcAAu2fX7IXdsRv2xN5/rVUVNQIte3QqNS63tzqOepbf/mUZdHrY+WL9qdlDERNCq0babYEEXag1vr9/9rI8lR2oDrJL9kz6L9gju6UOTP9VvVri2XNR3REcjl3RsSE0mDTjKsV8wlWqWptCMN28wIPOLFSEr4g5FT61J9CPJGX2E4NmLlY8GdjY50J/OqsjqfRoKrOUSU7PhMuOohd9GKZK45jGHBaRIy1FHOMEp9K8ZEm+VKmlSpGQ041vJh1+AJBdne4=</latexit>

Power
spectrogram(noise)⋮N0

Source model estimation
Separated
signalObserved
signal(18)

The parameters in this negative log posterior function L
are optimized by a maximum a posteriori estimation basedon the expectation–maximization (EM) algorithm [4]. Finally, multichannel Wiener filtering is applied to suppress
noise diffusing the observed signal using the estimated parameters.

To solve this problem, BS-ILRMA has been proposed.
The overview of BS-ILRMA is shown in Fig. 1, where00
W (noise)∈ CN ×M and W (mix)∈ CN ×M represent theiidemixing matrices for the noise samples x(noise)and the
ij 0
0
(mix)
(noise)observed signal xij , respectively. X m0
∈ CI×J
0and Y (noise)∈ CI×J represent the spectrograms of then0
m0 th channel in x(noise)and the n0 th channel in y (noise)
=
ij 0
ij 0
(noise)(noise) >I×J(yij 0 ,1 , . . . , yij 0 ,N 0 ) , respectively. X (mix)
∈
C
m
I×J
and Y (mix)∈Crepresent
thespectrogramsof then(mix)
mth channel in x(mix)
and
the
nth
channel
in
y
=
ij
ij
(mix)
(mix) >
.2(yij,1 , . . . , yij,N ) , respectively. | · | denotes entry-wiseabsolute and squaring operations, T n0 ∈ RI×L
≥0 represents

3. BS-ILRMA and Application to Hearing-Aid
System
BS-ILRMA has been proposed for a rescue robot that
detects a survivor’s voice in a disaster site [3]. BSILRMA simultaneously performs previously recorded egonoise training and extraction of the survivor’s voice. In

765

xxx
the shared basis matrix for the noise sources, T N ∈ RI×L
≥0
represents the unshared basis matrix for the target source,0and V (noise)∈ RL×J
and V (mix)∈ RL×J
represent the
n
n0
≥0
≥0
activation matrices for approximating Y (noise)
and Y (mix),0nnrespectively. BS-ILRMA employs two ILRMAs; one isapplied to the noise sample x(noise)to estimate W (noise)
and
i
ij 0
(noise)y ij 0 , and the other one is applied to the noisy speech
signal x(mix)to estimate W (mix)and y (mix)ijiij . The important
point is that the basis matrices for the noise sources, T n0 ,are shared between these two ILRMAs, and all the variables in these models are simultaneously optimized. Since
the shared basis matrices T n0 must represent similar spectra in both x(noise)and x(mix)ij , the noise spectral patterns willij 0
be captured by T n0 , and the other basis matrix T N willconsequently represent spectral patterns of the remaining
source, i.e., the target source.
The cost function of BS-ILRMA is defined as the sumof two cost functions of ILRMA as follows:( N0"#(noise) 2
X|yi,j0 ,n0 |
1 XX
(noise)+log
til,n0 vlj
J= 0
P
0 ,n0
(noise)
N
0v 0
t
0
il,n
0
0
l
lj ,n
n =1 i,jl)X(noise)− 2J 0log | det W i|preliminary experiment, we investigate the efficacy of BSILRMA for the hearing-aid system that we developed in
Sect. 4.1. Next, we evaluate the separation performance ofrank-constrained SCM estimation in which BS-ILRMA isapplied as the first step in Sect. 4.2.
4.1 Performance of BS-ILRMA
The performance of BS-ILRMA for the hearing-aid system is unclear because the conditions are different from thetask of separating ego-noise and target speech, e.g., small
available data length, long-distance from microphones tothe noise source, and various interference sources. Accordingly, we have to evaluate the performance of BS-ILRMA
for the hearing-aid system we developed. We comparedwith the following three methods: ILRMA, SS-ILRMA,
which separates a mixture using pretrained bases obtainedby noise samples, and BS-ILRMA, which optimizes theparameters for noise training and separation by sharing
bases. We recorded impulse response and diffuse noise ina room using the head-and-torso dummy we developed [8].
Figure 2 (a) shows the head-and-torso dummy, which simulates a person wearing a binaural hearing aid and holding
a smartphone. The head-and-torso dummy wears eight microphones (three microphones are attached to each ear [seeFigs. 2 (b) and (d)] and two microphones are attached to thesmartphone [see Fig. 2 (c)]), which are synchronized withthe same sampling rate. We numbered each microphone
as shown in Figs. 2 (b)–(d). Figure 3 shows the shape ofthe recording room and the positions of loudspeakers. The
reverberation time of the recording room is 300 ms. The
depth, width, and height are 6.5, 5.3, and 2.6 m, respectively. The distance from the head-and-torso dummy to theloudspeaker was varied by 75, 100, and 150 cm, and theangle was varied by −20, 0, and 20◦ , where 0◦ means thenormal to the head-and-torso dummy and the negative angle means the left side. The height of the head-and-torso
dummy was set to 170 cm, and the loudspeakers were set
in front of the head-and-torso dummy to mimic the situation of conversation. The target speech is a female utterance from the JNAS database [11] convolved with theimpulse response, which is down-sampled from 48 kHz
to 16 kHz. We provided a two-second noise period before the utterance for the training of SS-ILRMA and BSILRMA. The hamming window was used (64 ms in lengthand 50% overlap) in STFT. The observed signal was generated by mixing the recorded diffuse noise and the target
signal at the input SNRs of −10, −5, and 0 dB. The numbers of bases and iterations of ILRMA, SS-ILRMA, and
BS-ILRMA were set to 10 and 50, respectively. As thenoise pretraining for SS-ILRMA, the pretrained basis was
optimized by 50 update iterations. The observed signal
was preprocessed by sphering transformation by principal
component analysis. The demixing matrix was initialized
as the identity matrix, and the basis and activation matrices
were initialized by nonnegative random values. We usedsource-to-distortion ratio (SDR) improvement [12] as theobjective measurement criterion and averaged the scoresof each direction and ten initialization trials using different

i

+

( N0
"
1 XX
N

n=1 i,j

"
+

X
i,j

#

(mix) 2|yi,j,n|(mix)
l til,n vlj,n

P

+log

X

(mix)
til,n vlj,nl(mix) 2X
|yi,j,N
|
(mix)
+log
til,N vlj,NP(mix)
l til,N vlj,N
l

#

)
− 2J

X
i

log | det W (mix)
|
i

,

(19)

(mix)
where til,n0 , til,N , v (noise)lj,n0 , and v lj,n indicate the elements(noise)(mix)of T n0 , T N , V n0 , and V n , respectively. The update
rules of unshared parameters are the same as those in [2].
In contrast, since it is difficult to directly minimize Eq. (19)w.r.t. til,n0 , we designed the auxiliary function and minimize it to obtain the local optimal solution [3].
BS-ILRMA is more effective than ILRMA for a task
to separate the ego-noise and the target source [3]. However, applying BS-ILRMA to apply to the hearing-aid system is impractical because BS-ILRMA cannot be utilized
in a situation in which diffuse noise exists. Thus, we introduce rank-constrained SCM estimation. In this paper, we
propose the introduction of BS-ILRMA to the first step ofrank-constrained SCM estimation. BS-ILRMA is a linear
filter as well as ILRMA, and we can expect that the accuracy of the target speech extraction with the hearing-aid
system is improved by employing BS-ILRMA.

4. Experimental Evaluation of Proposed System
The purpose of this experiment is to evaluate the separation performance of rank-constrained SCM estimationin which BS-ILRMA is applied as the first step for therecorded data by our developed hearing-aid system. In the

766

xxx

2.8 m

3.3 m
6.5 m
Head-and-torso dummy

5.3 m

150 cm­20+20

100 cm75 cm

Head-and-torso dummy
Figure 2. (a) Overall view of head-and-torso dummy,(b) right-ear microphone array, (c) smartphone microphones, and (d) left-ear microphone array.

Figure 3. Shape of recording room and positions of loudspeakers (mouth position of conversation partner).
timation preprocessed by ILRMA, SS-ILRMA, and BSILRMA. We confirmed that rank-constrained SCM estimation clearly improves the SDR. In particular, rankconstrained SCM estimation preprocessed by BS-ILRMA
achieves the greatest average SDR improvement among themethods analyzed in almost all cases. Furthermore, thescores of rank-constrained SCM estimation depend on thescores of the preprocessing method. From the results,
we consider that BS-ILRMA is suitable for the parameter
initialization of rank-constrained SCM estimation.

random values.
The results are shown in Fig. 4. The improvementsof BS-ILRMA are greater than those of ILRMA and SSILRMA in almost all cases. From the results, BS-ILRMA
is expected to be effective for the preprocessing of rankconstrained SCM estimation.
4.2 Performance of BS-ILRMA Applied to
Rank-Constrained SCM Estimation
Next, we compare the performances when the rankconstrained SCM estimation is preprocessed by ILRMA,
SS-ILRMA, and BS-ILRMA. The experimental condition
was the same as that described in Sect. 4.1. The shapeparameter α and the scale parameter β for the inverse
gamma distribution, which is the prior distribution of rankconstrained SCM estimation, were set to 20 and 10−16 ,respectively. Since we have determined that the rankconstrained SCM estimation can achieve high separation
performance with few iterations, the parameters were optimized with only two iterations of the parameter update
calculation [8].
Figure 5 shows the results of rank-constrained SCM es-

5. Conclusion
In this study, we investigated the efficacy of BS-ILRMA
and rank-constrained SCM estimation preprocessed by
BS-ILRMA for our developed hearing-aid system. In
the first experimental evaluation, we confirmed that BSILRMA is effective in comparison with ILRMA and SSILRMA for the hearing-aid system in terms of separation
performance. The second experimental evaluation showedthat rank-constrained SCM estimation preprocessed by
BS-ILRMA can achieve high-quality separation performance. As a future work, we extend rank-constrained
SCM estimation to be a semi-supervised fashion.

767

xxx

SS-ILRMA

BS-ILRMA

ILRMA
SS-ILRMA
Rank-constrained SCM estimation (ILRMA Init.)
BS-ILRMA
SS-ILRMA
Rank-constrained SCM estimation (SS-ILRMA(ILRMA Init.)
Init.)
BS-ILRMA
Rank-constrained SCM estimation (SS-ILRMA Init.)
Rank-constrained SCM estimation (BS-ILRMA Init.)(a) 75 cm

20
15
10

SDR improvement [dB] SDR improvement [dB] SDR improvement [dB]SDR improvement [dB]SDR improvement [dB]SDR improvement [dB]

ILRMA

5
0

-10 dB20-5 dB(b) 100 cm

0 dB

15
10
5
0

-10 dB20-5 dB(c) 150 cm

0 dB

15
10
5
0

-10 dB-5 dB
Input SNR [dB]

0 dB

Figure 4. Average SDR improvements of ILRMA, SSILRMA, and BS-ILRMA under each input SNR condition.
Three figures show results when distance from head-andtorso dummy to target source is set to (a) 75, (b) 100, and(c) 150 cm, respectively.

(a) 75 cm

20
15
10
5
0

(b) 100 cm

20
15
10
5
0

(c) 150 cm

20
15
10
5
0

−10

−5

Input SNR [dB]0

Figure 5. Average SDR improvements of ILRMA, rankconstrained SCM estimation preprocessed by ILRMA,
SS-ILRMA, rank-constrained SCM estimation preprocessed by SS-ILRMA, BS-ILRMA, and rank-constrained
SCM estimation preprocessed by BS-ILRMA, respectively. Three figures show results when distance fromhead-and-torso dummy to target source is set to (a) 75,(b) 100, and (c) 150 cm, respectively.

ACKNOWLEDGEMENT
This work was partly supported by SECOM Science andTechnology Foundation and JSPS KAKENHI Grant Numbers 19H04131, 19H01116, and 19K20306.
REFERENCES[1] H. Sawada, N. Ono, H. Kameoka, D. Kitamura, and
H. Saruwatari, “A review of blind source separation
methods: two converging routes to ILRMA originatingfrom ICA and NMF,” APSIPA Trans. Signal and Information Processing, vol. 8, no. e12, pp. 1–14, 2019.

analysis for blind source separation,” in Proc.
EUSIPCO, pp. 1814–1818, 2019.
[5] N. Q. K. Duong, E. Vincent, and R. Gribonval, “Underdetermined reverberant audio source separation using a full-rank spatial covariance model,” IEEE Trans.
ASLP, vol. 18, no. 7, pp. 1830–1840, 2010.

[2] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and
H. Saruwatari, “Determined blind source separation
unifying independent vector analysis and nonnegative matrix factorization,” IEEE/ACM Trans. on ASLP,
vol. 24, no. 9, pp. 1626–1641, 2016.

[6] A. Ozerov and C. Févotte, “Multichannel nonnegative
matrix factorization in convolutive mixtures for audio
source separation,” IEEE Trans. ASLP, vol. 18, no. 3,
pp. 550–563, 2010.

[3] M. Takakusaki, D. Kitamura, N. Ono, T. Yamada,S. Makino, and H. Saruwatari, “Ego-noise reduction
for a hose-shaped rescue robot using basis shared semisupervised independent low-rank matrix analysis,” inProc. NCSP, pp. 351–354, 2018.

[7] H. Sawada, H. Kameoka, S. Araki, and N. Ueda, “Multichannel extensions of non-negative matrix factorization with complex valued data,” IEEE Trans. ASLP,
vol. 21, no. 5, pp. 971–982, 2013.

[4] Y. Kubo, N. Takamune, D. Kitamura, and
H. Saruwatari, “Efficient full-rank spatial covariance estimation using independent low-rank matrix[8] M. Une, Y. Kubo, N. Takamune, D. Kitamura,

768

xxx
H. Saruwatari, and S. Makino, “Evaluation of multichannel hearing aid system using rank-constrained spatial covariance matrix estimation,” in Proc. APSIPA,
pp. 1874–1879, 2019.
[9] Y. Takahashi, T. Takatani, K. Osako, H. Saruwatari,
and K. Shikano, “Blind spatial subtraction array forspeech enhancement in noisy environment,” IEEE
Trans. ASLP, vol. 17, no. 4, pp. 650–664, 2009.
[10] P. Smaragdis, B. Raj, and M. Shashanka, “Supervisedand semi-supervised separation of sounds from singlechannel mixtures,” in Proc. ICA, pp. 414–421, 2007.
[11] K. Itou, M. Yamamoto, K. Takeda, T. Takezawa,
T. Matsuoka, T. Kobayashi, K. Shikano, and S. Itahashi, “JNAS: Japanese speech corpus for large vocabulary continuous speech recognition research,” The
Journal of Acoustical Society of Japan (E), vol. 20,
no. 3, pp. 199–206, 1999.
[12] E. Vincent, R. Gribonval, and C. Févotte, “Performance measurement in blind audio source separation,”
IEEE Trans. ASLP, vol. 14, no. 4, pp. 1462–1469,
2006.

769