ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Authors: Zhongweiyang Xu
Xulin Fan
Zhong-Qiu Wang
Xilin Jiang
Romit Roy Choudhury
Venue: ICML 2025 poster (2025)
Keywords: Diffusion posterior sampling
speech separation
microphone array processing
Primary area: applications->language_speech_and_dialog
TL;DR: We show that we are able to apply diffusion posterior sampling for unsupervised, array-agnostic, and generative multi-channel blind speech separation.

Abstract: Blind Speech Separation (BSS) aims to separate multiple speech sources from audio mixtures
recorded by a microphone array. The problem is
challenging because it is a blind inverse problem,
i.e., the microphone array geometry, the room impulse response (RIR), and the speech sources, are
all unknown. We propose ArrayDPS to solve the
BSS problem in an unsupervised, array-agnostic,
and generative manner. The core idea builds on
diffusion posterior sampling (DPS), but unlike
DPS where the likelihood is tractable, ArrayDPS
must approximate the likelihood by formulating
a separate optimization problem. The solution to the optimization approximates room acoustics
and the relative transfer functions between microphones. These approximations, along with the
diffusion priors, iterate through the ArrayDPS
sampling process and ultimately yield separated
voice sources. We only need a simple single-speaker speech diffusion model as a prior, along
with the mixtures recorded at the microphones; no
microphone array information is necessary. Evaluation results show that ArrayDPS outperforms
all baseline unsupervised methods while being
comparable to supervised methods in terms of
SDR. Audio demos and codes are provided at:
https://arraydps.github.io/ArrayDPSDemo/ and
https://github.com/ArrayDPS/ArrayDPS.

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Zhongweiyang Xu
Xulin Fan
Zhong- Mhong-Qiu Wang? Xilin Jiang? Romit Roy Choudhury'
Abstract
ture their voices. Each microphone records mixture of all the speakers' voices. The objective is to separate these mix- tures to recover the individual voice sources. In recent years, supervised learning based methods have shown remarkable potential to solve the cocktail party problem (Wang & Chen, 2018). However, these methods are usually trained with supervision from speech datasets that were synthesized by using acoustic simulators. Such synthetic supervision inher- its several problems:
Blind Speech Separation (BSS) aims to sepa- rate multiple speech sources from audio mixtures recorded by a microphone array. The problem challenging because is a blind inverse problem, i.e., the microphone array geometry, the room im- pulse response (RIR), and the speech sources, are all unknown. We propose ArrayDPS to solve the BSS problem in an unsupervised, array-agnostic, and generative manner. The core idea builds on diffusion posterior sampling (DPS), but unlike DPS where the likelihood is tractable, ArrayDPS must approximate the likelihood by formulating separate optimization problem. The solution to the optimization approximates room acousties and the relative transfer functions between micro- phones. These approximations, along with the diffusion priors, iterate through the ArrayDPS sampling process and ultimately yield separated voice sources. We only need a simple single- speaker speech diffusion model as a prior, along with the mixtures recorded at the microphones; no microphone array information is necessary. Eval- uation results show that ArraYDSS outperforms all baseline unsupervised methods while being comparable to supervised methods in terms of SDR. Audio demos and codes are provided at: httpss/Arnnyydpssiiibbbbb///aa/
(1) Generalizability: The simulated dataset does not match real-world acoustic environments, causing model general- ization issues (Pandey & Wang. 2020; Zhang et al., 2021). (2) Deterministic: These models are trained to be determin- istic, and hence they output one fixed separation solution for a given mixture. This can give blurred results when the solution is not unique (Jayaram & Thickstun, 2020), i.e., the probability distribution of the sources, given the mixture, is multi-modal distribution.
(3) Fixed Array Geometry: These models usually assume the geometry of microphone arrays is fixed and known, pre- venting flexibility to unseen arrays (Taherian et al., 2022).
To address all the problems above, we propose Array generative, unsupervised, and arnay-agoostic algorithm for speech separation, which fully exploits speech prior in- formation. Building on the diffusion posterior sampling (DPS) technique (Chung et al., 2023b; Song et al., 2023), ArrayDPS treats speech separation as an inverse problem. Briefly, our goal is to recover speech sources from multi- microphone mixture measurements x A(s) + e, where A()) denotes the source mixing process in reverberant con- ditions. DPS samples from p(s|z) by using pre-trained diffusion prior p(s), and a tractable likelihood model p(rls). While we use a pre-trained speech source diffusion prior as well, unfortunately, the likelihood is intractable in our case. This is because the distortion function A(-) depends not only on the unknown array geometry but also on the unknown RIRs over which the speech arrives at each miero- phone. Without any knowledge of the array geometry, the RIRs, and the speech sources, this is referred to as "blind" separation problem.
and https///gttubb.oom////yypssssss/
1. Introduction
The cocktail party problem is a classic challenge in audio signal processing and machine learning (Cherry, 1953; Mc- Dermott, 2009). It arises when multiple speakers talk simul- tancously in the same room, and several mierophones cap-
'Department of Electrical and Computer Engincering, Univer- sity of Illinois Lubana-Champaiin Champaign, USA DDepartment of Computer Science and Engineering, Southern University of Science and Technology. Shenzhen, China *Columbia Univer- sity, NYC, USA. Correspondence to: Romit Roy Choudhury <croy @ illimois. edu>>, Zhongweiyang Xu <zx21 @ @illinois.edu>.
To solve the problems mentioned above, at each diffusion sampling step, with the current source estimate 8, we esti- mate A by: A arg max p(z A). Then we use p(x|
Proceedings of the 4 International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
A

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
as tractable approximation for the intractable likelihood p(z:))) Lastly, similar to DPS, can use the prior score and our approximated likelihood score to get the posterior score, which allows posterior sampling for separation.
(a) RIR signal
32
Model
h2a
haR
hyg
h22
g
We borrow ideas from Forward Convolutive Prediction (Wang et al., 202;a;b; Wang & Watanabe, 2023) for the estimation of A mentioned above, and we use Independent Vector Analysis (IVA) (Kim al., 2006; Hiroe, 2006) for initialization to improve sampling stability.
+++i $1,1
HH $2,7
+
112
$2,2
272
(b) Relative RIR Model
31
2
Extensive evaluation shows that ArrayDPS can achieve sim- ilar performance against recent supervised methods evalu- ated on ad-hoe microphone arrays, and performs the best among all unsupervised blind speech separation algorithms. The gains arise from using stronger speech priors from the diffusion models, as opposed to other unsupervised meth- ods (Wang& & Watanabe, 2023; Tran Vu & Cubch,, 2010; Kim et al., 2006) that do not fully exploit speech source priors. As a result, they often suffer from problems like frequency permutation or spatial aliasing, which pre- vents correct separation. In contrast, ArraDDSS leverages the diffusion prior which automatically bypasses those prob- lems.
11,1
h2.1
0
+
$1,1
#2,1
$1,2
8222
222
(c) Virtual Source Model
81
82
virtual source
81
82
# virtual source 2
Our main contributions are summarized as follows: Unsupervised: Only a clean-speech pre-trained diffusion model is needed, mitigating the generalization issues that affect supervised methods.
90-s1
g
0
-6-22
①
$1.1
$2,1
$1,2
$2.
Array-Agnostic: ArrayDS does not rely on array patterns and can generalize on any microphone array geometries. Generative: The method samples from the posterior, allow- ing multiple plausible separation results while fully exploit- ing the speech source prior.
2
22
Figure I. (a) Signal mixing in the real world; (b) Relative RIR model; and (c) Relative RIR from virtual sources to real channels. Measurement noise is ignored in the figures.
DPS for Multi-channel: ArrayDSS is the first method to solve the multi-channel array inverse problem with DPS; we believe that can enable many other applications beyond speech separation in multi-channel array signal processing,
where ht,,(t) (f) is the RIR from the 1eth source location to the th microphone, and is the convolution operation. (1)()) is the e microphone's 's white noise. Fig. (()) illustrates the signal model for the case of C' = 2 and K = 2. Our goal is to extract all K reverberant speech source images at the reference channel (c = 1), i.e., extract 81:K.1 from all the mixtures 11.C. without assuming any microphone array geometry, any source location, or any supervision'.
2. Problem Formulation
In a reverberant enviromment, assume C-channel micro- phone array is recording K concurrent speakers. Let de- note the K" clean speech sources as 51 (f), 82(t), (f..... & (t) e R. where E 1, 2,.,. T - 1 is the sample index of the waveform. These clean speech sources get filtered by the room impulse responses (RIR) and arrive at the C mi- crophones. forming the reverberant speech source images (k, E R. where k E 21, K} indexes the sources and e 21,2,..., C} indexes the microphones. Thus, each microphone captures a mixture of K source images and some measurement noise, and we denote this mixture as (.(t) E R. The seeaker-o--micrpphon filtering and mixing process can be modeled as:
We approach this problem through a generative model, i.e., sampling from p($1.K,1 [21c)) Before formulating that, we first discuss the relative room impulse response (relative RIR) model often useful for microphone arrays (shown in Fig, 1(b)). The relative RIR models the linear relationship between any given channel and a designated reference chan- nel (conventionally denoted as C₁ 1). Note that from Eq. 1, there exists a linear filter that can filter 8k.e (t) to
For convenicnce, we use skyikg.q to denote (sk,e(t) c1 S c C2 ky S k & k2), and same applies to all signals like T1.--
Sh.c (t) = ht,e(t) (t)
(1)
(e)( =
Sk. (t) + (t),
me(()~ N00, (2)
2

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Outpat: Sources
logp(11cc Outpur
i o -ffr 11 ( 11 #c))dr
ODE Solver
(Sampling ODE)
$111 * +
f2.1
812
+
622
Source 2 Sumpling ODF
Source Sampling ODF
+irnt+ +
62.0
#+
BC
Vop lonp(+t) oo logp(r 12... Folti
Convolve
prior soore
likeishood score
og Likellhood Approsimation
Canvolve
log legp($1cclf..3
DifTusion U-Net
Backprop. of
Gi-t w Wareeooeon
Elp #1003
Noisy Sources (bxthaxee of
denoise
M
Input Mutti-hhandl Mistures
Hobntt
49
Relative Estimator
Log Likelihood Approx. 20 1f
80
arg max log p( J:K
(1a)
#1.40
21:
FC-
(a) Array DPS Overview
(b) Zoom-in View of Log Likelihood Approximation
Figure 2. An overview of ArmaDDS for K = 2 sources. Left figure (a) shows the pipeline for separation with diffusion posterior sampling, which uses the likelihood approximation pipeline shown in the right figure (b).
obtain Sk (t) if hg,ea (t) is invertible:
3. ArrayDPS
k Deg -5 og (t) hx.c2 (t) ho,oo (t)
(3)
3.1. Brief Background and Overview
Sk.og (t) Dcs -ey (t) Sk.cy (t)
(4)
Background: Our problem is formulated as T1.C A(1i.) + n1..c where A can be understood as the filter and sum process in Fig. 1(c). Separating the sources can be viewed as sampling from the posterior p(81-k (11c). For this, use Diffusion Posterior Sampling (DPS) (Chung et al., 2023b; Song et al., 2023; 2021) which samples from the posterior by solving a score-based probabilistic flow ordinary differential equation (ODE):
Eq. 3 is called the relative RIR between channel c and 02 for speaker k. and Eq. 4 is the convolution filtering pro- cess. The relative RIR model is useful because the reference channel can serve as an anchor, and all the other channel measurements can be modeled relative to this anchor.
Can we take one more step and avoid designating micro- phone as reference channel? One idea is to imagine virtual source Sy for each speaker and apply a relative RIR filter to map it a real channel. Fig. (c) shows an example where relative RIR 96-11 (t) maps the virtual source si (1) to the source image s (1) at channel 1. The same occurs for all (virtual source, real channel) pairs:
(6) where of is the noisy version of s at diffusion time т. The ODE is adapted from EDM (Karras et al., 2022), assuming linear noise schedule (a(T) = T). Score-based diffusion model (Song et al., 2021; Karras et al., 2022) suggests that sampling from p(s (z1.c) is basically first initializing the sources Tau ~ (O, o² (Tmax) )). and then getting the sepa- rated virtual sources $1:K by solving the ODE in Eq. 6 until Tiin ~ O. We elaborate on the details in Appendix A.
dsi = -o((TV v
(t.c(t) = - st))), CE C}
(5)
What is the advantage of modeling such a virtual source? The key reason is that virtual sources (along with their rela- tive RIRs) offer the flexibility to model many system config- urations. For instance, a virtual source could be the anechoic speaker's ss voice signal, or it could be the measurement real channel CI, it could even be the signal at imag- inary microphone co placed near the speakers. Whatever the virtual signal represents, the corresponding relative RIR filters can adapt to match the measurements FT.C at the real microphones. This flcxibility allows (1) treating the refer- ence channel e the same as all other channels, and (2) more freedom in the optimization, allowing better performance in some algorithms (Wang& & Watanabe, 2023).
To acquire the posterior score V logp(s (an.c) in the ODE above, DPS decomposes the posterior score into a prior score and a likelihood score, where the former is estimated from a pre-trained diffusion model, and the latter is approximated based on a tractable likelihood model (also explained in Appendix A). However, in our "blind" source separation problem, the likelihood is intractable, and solving this is at the crux of this paper.
Overview: Fig. 2(a) shows the ArrayDPS model. The multi multi-channel audio mixtures T1.C are inputs (shown at the bottom left of the figure) and the outputs are separated vir-
3

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
tual sources from the ODE (shown the top of the figure). The whole Ars pipeline essentially develops the com- ponents needed for the ODE to reverse the diffusion process. At each diffusion time т, ArrayDPS can be summarized in five key steps (following Fig. 2(a)):
(STFT) domain. In STFT domain, Eq. 5 becomes:
Sk,e f) Go)s((f) (l.
(8)
where Sk,c(1, f) and Sx f) are the STFT of (t,c(t) and su (t), respectively, (f) is the discrete Fourier trans-
(Step 1): The final posterior score Vsi.x I (1c)) in the ODE is constructed by adding the prior score Veg log p(sT)) and likelihood score log V=(p(21.l(5..)
form of ( where the FFT size is the same as the STFT.
Relaxing narrowband assumption: The above equation is unrealistic for speech signals because it makes a narrowband approximation: it assumes that the length of the relative RIR filter is shorter than the FFT size. In real environments, the relative RIR filter can be in hundreds of milliseconds, while the FFT size is usually only tens of milliseconds. To relax this assumption, we model the relative RIR over multiple time-frames as Gb (f:
for each source.
(Step 2): The prior score is estimated using Tweedie's For- mula: V V=rp($])= (og- (T). The &% in the formula, which is the estimate of clean source, is the output of a dif- fusion denoising U-Net with noisy source si as input. (Step 3): The likelihood p(21,c181:x ($1:) is intractable be- cause the distortion function A(-) (filtering+mxxing) in- cludes the unknown relative RIRs. However, given the relative RIRs, the log likelihood log p(z1cc) $9.K Blik.ghik... 1o)) becomes tractable. This motivates our log likelihood ap- proximation model expanded in Fig. 2(b).
f). This multi-frame filter in the STFT domain is then convolved with the virtual source as follows:
(9) where 1, are frame and frequency index, and F, P are the number of future and past frames of the relative RIR
G fc(
(l, all in STFT domain. Observe that *1 in Eq.9 denotes convolution in the frame dimension:
Given the measured mixture 21.C and the denoised esti- mates &ix from time T, the yellow module in Fig. 2(b) es- timates the relative RIRs 90-11:c in maximum likelihood manner (detailed next in Sec. 3.2). Recall these relative RIRs are transfer functions from the virtual sources to the real microphone channels, hence appropriately convolving 00-Mic with the virtual source estimates $1.K gives us the actual source image estimates $1:K.I:C at all the microphone channels. Adding up these actual source image estimates yields the estimated mixtures a1.0. Finally, based on the Gaussian noise model (.,(() in Eq. 2, the log likelihood can be directly calculated as ||æ1.c 2 12 weighted by signal energy-based -ased scaler ç. (Step 4): Back in Fig. 2(a), the approximated log likelihood is back- prooaprop to 8x to compute the likelihood score. (Step 5): This likelihood score and the prior score (from step 2) are combined obtain the posterior score at T.
ch,o(1 *1 Ss(l Σ Gh..d. f. (10)
Thus, the complete signal model - that connects the virtual sources the actual mixtures the microphones can be modeled as:
K Xe(l,)) Σ fl Su N) A=1
(11)
ML Estimation: Under this setting, we intend to estimate relative RIRs (4, in a maximum likelihood manner. We found that the Forward Convolutive Prediction (FCP)
method (Wang et al., 2021a;b; Wang & Watanabe, 2023) is able to achieve this. Basically, FCP estimates relative RIRs G f) from the multi-channel mixtures Xo(1, f)
and the virtual source estimate (x(1, by solving linear problem:
Note that Fig. 2(a) is separating the virtual sources $1.K- Since we want to estimate the source images at the reference channel, namely 81:K,1, we convolve with the estimated relative RIRs o to obtain the separated source images 1k,1-
G5 = ().(
(12)
argmin Σ
[xo(1.) - (l. Su
GB 1f 1b(1,)
(13)
1 G 15a, = Σ x.l. + c. max C Σ
1
x.U. DP (14)
3.2. Relative RIR Estimation
Since log likelihood log p(21,c16 (0-1:0)) is tractable, we propose to estimate the 1:K in maximum likelihood manner using mixtures F1.C and denoised sources $9.x:
1:K
As shown above, FCP is solving a weighted least squares problem, so it has an analytical solution as in (Wang & Watanabe, 2023). The weight X f) aims to prevent over- fitting to high-energy STFT bins, and is a parameter to adjust the weight. Note that two important parameters for FCP are F and P from Eq. 10, which control the past and future filter lengths of the relative RIR. We prove FCP is equivalent to the maximum likelihood relative RIR estima- tor in Appendix B.2 Theorem B. Thus, with the FCP estimated filters, we can obtain the estimated source images
I:K 90-11.c
1:K arg max log p( (o $P.k)
s0-110 However, previous work has shown that it is better to esti- mate the filters in spectral domain (Saijo et al., 2024; Gan- not al., 2017) instead of in the time domain shown above. Hence, we transform to the Short- Time-Fourier- www.--oo-urrrrrooooiooo Transform
4

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
at all microphone channels:
Algorithm ArrayDPS
f1, = fl) (1,
(15)
Require: {N, (o...... (u. )
Inputs: mixtures T1:C, number of sources K
3.3. Posterior Score Approximation
1: X1.C <- -TTFT(IICC)
The log likelihood approximation gives relative RIR esti- mates and the corresponding source image estimates at each microphone channel. All of these were computed based clean source estimates denoised by the diffusion model at time r. Using all the available estimates, ArrayDPS now needs to compute the posterior score, and then run the ODE solver in Eq. 6 to ultimately get samples from p( (2c))) We describe this process next.
$11 .............................. SK K. + (VA(X
IVA separation SIVA K.
3: sM ... K,1 + (-1, ).1):.... ..... (
4: for k = 1 to K, c 1 to C do
5: Gf + FCP(Xc., SIVA)
SM
IVA Init. relative RIR
6: end for
7: for l = 1 to K do
8: Initialize 8 ~ (8 od Diff. Init. with IVA
end for
10: for = 0 N do
& order Heun stochastic sampler
Assume we have pre-trained diffusion denoising model
11: ã; <- σ; 7700
(o(% (T)) trained on single-channel speech sources;
12: Sample 6: ~ (0, Snoise!) !)
the corresponding score model (s., (T)) approximates
13:
for k to K do
Vor log p( Based on this, we present here a novel ap-
14:
% + s + v 0 ofe
add stochasticity
15:
end for
proximation of the posterior score V x Mogp(&I.K /m..c)
16:
score <- CET, CET,,CORE (8 TFC. C i)
D Algo.2
to enable sampling using the ODE above. All the derivation,
17:
di - score
proof, and analysis of our approximation is in Appendix B. Below is our final result, where we first denoise the noisy sources xi.x and then transform to STFT domain:
18:
$1.k 8l.k + (ar41 à.) .i order Euler step
19:
if # then
b 2nd order correction
20:
score' + -ET GBT_scORE(s T++1
1:K 31:0 G (+ 1)
# = (o(s),a(TT)
S2 STFT(EE), A E (1....,
K}
21:
d' <- - 0i+1' score'
(16)
22:
81.K
3+11 +- i + [ 6.) (d. + d))
Then we use FCP to estimate all relative RIRs:
23:
end if
24: end for
Go = Xx,( 5 Sx.c= Gb-re * Sy
(17)
25: ST.K + STFT($ N
The relative RIRs enable log likelihood estimates (at the output of Fig. 2(b)), and to get their respective likelihood
26: for k = 1 to K do
P transform to channel 1 with FCP
27: GR-tin +- FCP(X1. 1.P(X).S.. S
scores, we back-propagate to 8i. s8 to calculate the gradient.
28: $u.1 + (STFF(GG
S4)
In the final step, the likelihood scores and the prior scores are summed up to give the posterior score:
29: end for
30:
return 8Tix- A1:K, separated virtual and channel sources
K
V log logp(six(211c)
So(sL, +
initialization purposes, we use IVA to separate the sources at the reference channel (c1 = 1) and then use these separated sources to further estimate the relative RIRs G1
c VV sR E(T)
K Σ
Ite ISTFT
Sk.
(18)
51,1,
GIVA
SK IVA(X1.c)
(19)
ISTFT above denotes Inverse-Short- Time-Fouries Transform. Note that the functional relationship between .i.K and 50 L.K.IIC is fully differentiable, as operations Do. STFT, ISTFT and FCP are all differentiable. Thus, in the second term in Eq. 18, the likelihood score can be si.k- Again, detailed proof of correctness, analysis, and relation to Expectation Maximization is included in Appendix B.
G.EIVA
FCP(X SKA) k (1,-.,, K], CE (1, C
(20)
SIVA Then the IVA separated sources Si:K.1 are used to initial- ize the starting point of posterior sampling, and the esti- mated relative RIRs G*IIA are used to guide the posterior sampling in early stages, which will be discussed next in Sec. 3.5.
calculated by back-propagating
3.5. Algorithm
3.4. IVA as Initialization
Our proposed ArrayDPS is specified in Algorithm 1, which calls the posterior score approximation function, GRT SCORE (-), presented in Algorithm 2. We intuitively explain blocks of the algorithm below while the details can be found in Appendix C.
At the early stages of solving the separation ODE, the source estimates $9.K are extremely inaccurate. which means the estimated relative RIRs are inaccurate. To mitigate this, we propose to use an Independent Vector Analysis algo- rithm (Kim et al., 2006; Hiroe, 2006) as initialization, for both the sources and the relative RIRs.
The ArrayDPS algorithm in Al- gorithm 1 needs a few sampler parameters to be pre- defined: sampling steps and schedule N, oie
ArrayDPS Algorithm:
IVA is a classic unsupervised algorithm for multi-channel blind speech separation. which has limited performance in reverberant conditions (Kim et al., 2006; Hiroe, 2006). For
and Tie (O....,N.N11. Smose, which all follow the stochastic
5

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Algorithm 2 Posterior Score Approximation
in Appendix C. The algorithm takes the current diffusion noisy sources Si.K- the mixtures 11:C. the IVA initialized relative RIRs G and the diffusion step as inputs. In lines 2-6, the noisy sources are denoised, and are then used to estimate the prior score (using Tweedie's 's Formula). Then in lines 9-15, the relative RIRs (estimated from IVA init. or estimated using current source est.) are used to transform estimated virtual sources to all channels. In line 16 the ap- proximated likelihood score is estimated. Line 17-19 is the reference channel guidance mentioned. Then in line 21, the posterior score is the sum of the prior score and likelihood score.
Require: {Da, Ii((u....N) NNef, Nig, E2())
1: function GET GET_SCORE FL.C. G1-c. i)
2: for k- to K do
3:
$g - Do o.)
diffusion denoiser
4:
SR < (T)
5:
end for
fi. fik
6:
C: -
of XI.C STFT(11.))
P get prior score
7:
8:
for k 1 to c= to C do
9:
ifi Ng then
10:
Gb + FCP(Xe; $2)
b Est. relative RTR
11:
else
12:
_o + GAAVA
> Use IVA init. relative RIR
4. Experiments and Evaluation
13:
end if
14:
Ss.c 4- ISTFT(CU--C t1 52)
filter all channels
We first discuss ArrayDPS's training and sampling config- urations, well as the baseline configurations. Then show results and analysis on both fixed microphone array (test samples are all from one single microphone array) and ad-hoc microphone arrays (test samples are from differ- ent unknown microphone arrays). We have open sourced ArrayDPS in
15:
end for
16:
g (I(T.VV 1.x
Σ
Ie- Σt=1 $
Sk.c
17:
if S then
> reference channel guidance
K h- 20 :
18:
E2(Ti) TI
19:
9: g Ar=
b update posterior score
20:
end if
21:
return +
> final posterior score
4.1. Datasets
22:
end function
As mentioned in Sec. 3, ArrayDPS relies on pre-trained single-channel single-speaker denoising diffusion model Dg. We train this unconditional speech diffusion model on a clean subset of speech corpus LibriTTS (Zen et al., 2019). Since the virtual source could be reverberant, we train another diffusion prior to reverberant speech, where we convolve LibriTTS clean speech with room impulse re- sponses (RIR). We discuss details, including the architecture and diffusion training configurations, in Appendix D. For the final sampler and the posterior score approximation, we discuss the parameter configurations in Appendix C.3.
sampler in EDM (Karras et al., 2022). The algorithm then takes the mixtures F1.C and number of sources K as in- puts, to separate source images in the reference channel. From lines 1-6, IVA is used to separate reference-channel source images, which are further used to estimate the rel- ative RIRs. Then in lines 7-9, the IVA separated sources are used as diffusion initialization. Line 10-24 is a second- order Heun stochastic sampler following EDM (Karras et al., 2022), where the score is using our Posterior Score Approx- imation, GET_SCORE (-), in Algorithm sampler is sampling the virtual sources instead of reference channel source images; we discuss the reason later in Ap- pendix G. Then, to output separated source images in the reference channel, lines 25-30 use FCP to estimate the rel- ative RIRs that allow filtering virtual sources to reference channel source images.
Note that the
For evaluation, we use SMS-WSJ (Drude et al., 2019b) dataset for fixed microphone array evaluation and use Spa- tialized WSJO-2Mix dataset (Wang et al., 2018) for ad-hoc microphone array evaluation. SMS-WSJ contains simulated mixtures and sources fixed microphone array while Spatialized WSJO-2Mix contains samples on different mi- crophone arrays. All datasets are designed for 2-speaker source separation. Details of the datasets are explained in Appendix E.
Posterior Score Approximation Algorithm: The posterior score approximation in Algorithm 2 follows the result in Sec. 3.3 and provides Algorithm 1 with the posterior score for diffusion sampling. Algorithm 2 needs a few pre-defined parameters. Do is pre-trained diffusion denoiser. Nia is the number of early steps needed for reference channel guid- ance, where the sum of estimated sources are directly guided by the reference channel mixture (lines 17-19). We found this empirical term helps improve algorithm robustness and convergence. Nig means in the first Ni, sampling steps, the IVA initialized relative RIRs are used to calculate the like- lihood, instead of the FCt-estimated ones. (2()), X are all weighting scalars which will be discussed in detail
4.2. ArrayDPS and Variants (for ablations)
Row 2a-2e, 3a-3b, and 4a-4b in Table 1 show Array- DPS and its variants for ablations. ArrayDPS-A in row 2a is default ArrayDPS with anechoic clean speech diffu- sion prior. ArrayDPS-B in row 2b is the same but the diffusion prior is trained on reverberant speech. ArrayDPS- C in row 2c shows the ablation when the reference channel guidance is removed from Algorithm 2. ArrayDPS-D in row 2d shows the ablation when IVA initialization is re-
6

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Table 1. Evaluation results for 3-channel SMS-WSJ. Methods denoted with results from UNSSOR (Wang & Watanabe, 2023), and methods denoted with * mean it is impractical. Note that SMS-WSJ only contains samples with fixed microphone array Top results are
color-coded as top1 top2
and top3
Row
Methods
Unsup.
Arry Agnostic
Prior
IVA
Ref.
SDR (dB)
SI-SDR (dB)
PESQ
eSTOI
Init.
Guide.
0
Mixture
0.1
0.0
1.87
0.603
la
Spatial Cluster
9.5
8.5
2.52
0.759
lb
IVA IVA
Laplace
12.0
10.7
2.67
0,802
lc
Gaussian
13.4
12.2
2.82
0.834
ld
UNSSOR
15.4
14.4
3.20
0.875
Za
ArrayDPS-A A
Anechoic
15.8+1.2
15.0+1..3
3.38-10.12
0.865=0.020
2b
ArrayDPS-B B
Reverb,
15.1+1.1
14,3+112
3.29++0.1
0.850=0.019
2c
ArtayDPS-C
Anechaic
14.9-1..3
14.0
3.16-10.11
0.844-0.022
2d
APS D
Anechoic
8.544.6
6.815.2
0.731=0.111 2.59
2e
ArrayDPS-E E
Anechoic
X
0.9 x
-144199
1.74+0.15
0.518=0.058
3a
ArrayDPS-A-Max
Anechoic Anechoic
17.2
16.5
3.52
0.888
3b
ArrayDPS-D-Max
14.4
13.4
3.19
0.860
4a
ArrayDSS A- A-ML
Anechoic
16.9
16.2
3.49
0,884
4b
AraayDPS- (mayDDS-D-MML. M-M..
Anechoic
14.0
12.9
3.15
0.851
5a
TF-GridNet-SAe S
X
16.8
16.3
3.91
0.924
5b
TF-GridNet-SMS
X
16.2
15.7
3.72
0.908
5c
TF-GGriNNe--Satiall
(3 (3-mics)
14.7 14.1
3.35
0.877
moved, while ArrayDPS-E in row 2e shows the ablation when both initialization and reference channel guidance are removed. Note that the sampling parameters for Ar- rayDPS with and without initialization are different (see Appendix C.3). Because all ArrayDPS Row 2(a-e) are generative, we sample 5 separated samples for each mix- ture and report the 5-sample mean and standard deviation ste in Row 2, Table 1), averaged over all the mix- tures in the test set.
Note that Sp-atialet-Spatiall in Table I row 5c should gen- eralize to any 3-channel ad-hoc microphone arrays. All baseline models are discussed in detail in Appendix F.
4.4. Metries
For metrics, use the first channel clean source images as the reference signals. We use the signal-to-dise -odistoronitto ratio (SDR) (Vincent et al., 2006) and Scale-Invariant SDR (SI-SDR) (Roux et al., 2018) to measure sample-level sepa- ration performance. We use perceptual evaluation of speech quality (PESQ) (Rix al., 2001) and extended short-time objective intelligibility (eSTOI) (Taal et al., 2011) mea- sure perceptual quality and speech intelligibility respectively.
In Row 3a-3b, ArrayDPS-A-Mar and ArrayDPS-D-Me shows the max metric score from the samples, averaged for all test samples. Row 4a-4h represents ArrayDPS-A-MI. and ArrayDPS-D-ML, where ML stands for 'maximum likelihood'. Basically, from the samples, we find the one with the maximum likelihood (highest mixture reconstruc- tion SNR) and report the separation metric averaged over the test set. Row 3a and 4a's methods are the same as the default 2a (default), while 3b and 4b are the same as 2d (no initialization). Note that Row 4 a practical algorithm because only uses the mixture to calculate the likelihood.
4.5. Fixed Microphone Array Evaluation
We show the evaluation result for the 3-channel (using first, third, and fifth mics) SMS-WSJ test set in Table 1. 6-channel SMS-WSS results are in Appendix I.
We first compare against the unsu- pervised methods. Comparing rows 2a vs. (a--d), we see ArrayDPS-A's mean score consistently outperforms spa- tial clustering and IVA-based methods by substantive margin; also outperforms the unsupervised state-of-the-art method UNSSOR in all metrics except eSTOI. Moreover, ArrayDPS-A' 'ss standard deviation is quite robust. Note that UNSSOR is trained only for fixed array on the SMS-WSS dataset. If we further compare row 4a vs. 1d, we see that by selecting the maximum likelihood sample, ArrayDPS-A ML shows strong improvement over UNSSOR (1.8 dB of SI-SDR 0.29 PESQ).
Against Unsupervised:
4.3. Baseline Methods
For the baseline models, we include three unsupervised models: Spatial Clustering (Rickard, 2007; Tran Vu & Haeb-Umbach, 2010; Sawada et al., 2011; Ito al., 2016), Independent Vector Analysis (IVA) (Kim et al., 2006; Hi- roe, 2006), UNSSOR (Wang& & Watanabe, 2023), as well as supervised TF-GridNet (Wang et al., 2023). Te-GridNet- SMSt is the supervised baseline model reported in UN- SSOR (Wang & Watanabe, 2023) trained on SMS-WS WSJ Corpus (Drude et al., 2019b). TFS and TF- GridNet-Spatial are our reproduced TF-GridNet trained SMS-WSI and Spatialized WSJ0-2Mix corpus respectively.
Against Supervised: Outperforming recent supervised
7

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
methods is extremely challenging and all previous unsuper- vised methods exhibit gap to supervised methods (Wang & Watanabe, 2023; Saijo et al., 2024). On comparing row 2a vs. 5a-5b, ArrayDPS-A is consistently worse than the recent supervised methods, as expected. However, compar- shows better SDR
Table 2. WSJO-2Mix. Note that the microphone positions are random for this dataset. Top results are emphasized in topl top2
Ad-hoc Array Evaluation results for 4-channel Spatialized
and
top3
Methods denoted with * mean it is impractical.
Row
Methods
SDR
SI-SDR
PESQ
eSTOI
ing row 3a vs. 5a-5b, ArrayDSS-A and SI-SDR than the supervised method, which means if we sample 5 samples using ArrayDPS, one of the samples can outperform supervised methods in terms of SDR and SI-SDR. However, since ArrayDPS-Max 's max operation is not practical, we check ArrayDPS-A-ME i.e., row 4a vs. 5a-5b; observe that the maximum likelihood sample from 5 samples achieves slightly better SDR than the supervised method, but a bit worse on other metrics.
Mixture
0.2
0.0
1.81
0.545
la
Spatial Cluster
9.3
8.0
2.48
0.745
IVA-Laplace
7.9
5.2
2.41
0.648
IVA-Gaussian
12.5
10.1
3.01
0.808
ld
UNSSOR
15.2
14.2
3.54
0.873
2a
ArrayDPS-A
16.1
15.3
3.47
0.877
+0.6
+0066
+0.07
+0.012
3a
ArrayDPS-D
6.7
4.7
2.45
0.677
+4.5
+5.0
+0.51
+0.120
4a
ArrayDPS-A A- A-Max*
16.8
16.0
3.54
0.891
Ablation Studies: We compare the ablations in rows 2a- 2e. Comparing 2b vs. 2a, we find that it is better to use the anechoic clean speech diffusion prior instead of the reverberant one. The possible reason is related to FCP, which we explain in Appendix G. Comparing 2c vs. 2a, we find that with initialization, the reference channel guidance can improve all metrics and reduce the standard deviation (instability). Comparing 2d vs. 2a, the performance drops severely without the IVA initialization, while the std is also high, meaning sometimes the method can separate but not always. When it's able to separate, from row 3b and 4b, the best result from 5 samples is reasonable even without IVA initialization. Lastly, we compare row 2d vs. 2e and find that without IVA initialization, the reference channel guidance extremely important to obtain reasonable performance.
4b
ArrayDPS-D4 -
12.9
11.6
3.14
0.830
5a
ArrayDPS-A-Mh A-ML
16.6
15.8
3.52
0.886
5b
ArrayDPS-DDMM 12.3
10.8
3.06
0.810
6a
TF-GridNet-Spatiin
15.8
15.1
3.67
0.895
and SI-SDR, showing ArrayDPS's superiority in ad-hoc array setting.
4.7. 3 speakers and Real-World Evaluations
We also show that our DPSwoDsS nicely in 3-speaker and real-world samples. For 3-speaker evaluation, we evaluate ArrayDPSon the Spatialized WSJO-3Mix dataset, which is the 3-speaker version of the Spatialized WSJO- 2Mix dataset. This means the dataset also contains samples recorded from ad-hoc microphones. The separation results are shown in Appendix J. showing that ArrayDPS easily outperforms supervised methods by a large margin.
Lastly, we check row 5c, which is supervised T'rGrriNNet trained on 3-channel Spatialized WSJO-2Mix dataset (ad- hoc array dataset) and we test how well it works on the SMS- WSJ dataset. Observing row 5b vs. 5c, we see that for su- pervised models, generalizing to ad-hoc arrays costs a huge performance drop. Comparing row 5b vs. 2a, ArrayDPS- outperforms supervised TF- TF-GridNet-Spatte in SDR and SI-SDR by about IdB, while ArrayDPS-A-MS (row 4a) outperforms STatriaNet-Spatiall for all metrics.
For real-world mixture evaluation, we recorded 15 mixtures in a 7m x 4m 2.7m room. Two volunteers speak simul- taneously with weak environmental noise and are recorded by 3 microphones. The separation results are shown in the demo site: https:/arrraydps.ginnn http:/ppp.giiuuubbo///raaa DPSDemol.
4.8. Source and Filter Visualizations
Recall that DPSaappssssples virtual source signals and then uses FCP to estimate the relative RIRs that filter the virtual sources to the reference channel sources. Thus, we conduct visualization analysis of the virtual sources, and the es- timated relative RIRs are in the spectrogram domain. The visualization and a detailed explanation are in Appendix L. The virtual source listening demos are also on our demo site. Our observation is that the virtual source separated is closer to the anechoic clean source speech than the reverber- ant clean source speech, meaning that ArrayDPShas some dereverberation effects.
4.6. Ad-hoc Array Evaluation
We evaluate the ad-hoc array setting on the Spatialized WSJ0-2Mix (Wang et al., 2018) dataset. In Table 2, we show results for the 4-channel case (using the first 4 mics for each sample, different samples' array geometries are different); the 2-channel and 3-channel results are in Ap- pendix H.
Following Table 2, ArrayDPS-A-ML (row 5a) outperforms all unsupervised methods by substantive margin, while ArrayDPS-A (row 2a)'s mean metrics performs better than UNSSOR (row 1d) in SDR, SI-SDR, and eSTOI. Com- paring row 2a vs. 6a, we see AraaDDPS-A's 's mean metric scores are even better than supervised GriGrid in SDR
5. Related Work
Blind speech separation has been advanced greatly by deep neural networks using supervised training (Wang & Chen,
8

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
2018: Hershey et al., 2016; Yu et al., 2017; Kalkhorani & Wang, 2024; Wang et al., 2023; Quan & Li, 2023; Tahe- rian et al., 2023; Xu et al., 2024; Li et al., 2024). However, supervised methods have generalization issues (Pandey & Wang, 2020; Zhang et al., 2021), especially in multi-channel scenarios, where training on multiple microphones with ar- ray agnostic tricks (Luo et al., 2020; Yoshioka et al., 2022: Taherian et al., 2022) cannot guarantee generalization on unseen microphone arrays (Taherian al., 2022). Also, supervised methods are usually discriminative, often pro- ducing blurred results with nonlinear distortions. To solve this problem, generative separation methods (Subakan & Smaragdis, 2018; Scheibler et al., 2023; Karchkhadze et al., 2024; Lutati et al.) are proposed to sample clean sources conditioned on the given mixture. However, these generative methods still need clean-mixture paired data for supervised training.
to generate from distribution by following a probabilistic SDE/ODE from a noise initialization (Song al., 2021). Diffusion posterior sampling (DPS) (Chung et al., 2023b; Song et al., 2023; 2021) tries to solve the inverse problem with a pre-trained prior diffusion model and a known like- lihood model. Later, a few works are further proposed to solve the blind inverse problem with DPS (Chung et al., 2023a; Moliner et al., 2024b; Bai al., 2024; Svento et al., 2025b; Thuillier et al., 2025; Svento et al., 2025a; Moliner et al., 2025; Laroche et al., 2024).
More recently, DPS has been applied for source separa- tion (Yu et al., 2023; Mariani et al., 2024; Jayaram & Thick- stun, 2020). However, these methods only consider single- channel source separation where the likelihood is tractable, which not the case for multi-channel problems.
6. Conclusion
For unsupervised separation, independent component anal- ysis (ICA) (Amari et al., 1995; Hyvärinen et al., 2009; Sawada et al., 2019) separates sources by enforeing inde- pendence between sources. Independent vector analysis (IVA) (Kim al.. 2006; Hiroe, 2006; Nakashima et al., 2021) adds Gaussian assumption on each source's STFT bins. Spatial Clustering (Tran Vu & Hacb-Umbach. 2010; Sawada et al., 2011; 1; Ito et al., 2016; Rickard, 2007) based methods separate the sources by clustering the STFT bins using spatial features.
This paper proposes ArrayDPS, an unsupervised, array- agnostic, and generative method to separate speech from multi-channel mixture recordings. ArrayDPS proposes to use a speech diffusion prior and a novel likelihood approxi- mation to enable posterior sampling. The result shows our method outperforms other unsupervised methods and per- forms on par with supervised methods in SDR. We leave future work to explore ArrayDPS for more general array inverse problems.
Deep learning based unsupervised source separation has also been studied widely. Mixture invariant training (MixIT) (Wisdom al., 2020; Tzinis al., 2020; 2022a;b; Saijo& & Ogawa, 2023; Sivaraman et al., 2022) synthesizes new mixtures by mixing real-world mixtures, and train the separation model output sources can be mixed somehow to the original mixtures. MixIT has been modified to the multi-channel scenarios (Wang & Watanabe, 2023; Han et al., 2024), but shows limited performance. Another line of work (Tzinis et al., 2019; Drude et al., 2019a; Seethara- man et al., 2019; Togami et al., 2020; Xu & Choudhury, 2022) uses classic methods like spatial clustering to generate pseudo-labels as training targets. However, the performance is bounded by the classic method. More recently, one group of work tried to exploit the multi-channel signal model for unsupervised training (Aralikatti et al., 2023; Saijo et al., 2024; Wang& & Watanabe, 2023; Drude et al., 2019c; Bando et al., 2021; 2022). These methods train neural network to separate sources, which after filtering and mixing, would be close to the multi-channel mixture. Although these methods show promising results, they do not exploit the speech prior information during training.
Acknowledgment
We thank Foxconn and NSF (grant 2008338, 1909568, 2148583, and MRI-2018966) for funding this research. We are also grateful to the reviewers for their insightful feed- back.
Impact Statement
This paper explores advancements in unsupervised, array- agnostic, and generative blind speech separation, which lies in the intersection of several established fields, including ar- ray signal processing and machine learning. The algorithm we developed has many positive societal consequences. One notable application of our algorithm is assistive/augmented hearing, which improves communication and accessibility for individuals in noisy environments. especially those with hearing impairments. Also, our method has the potential to improve automatic speech transcription technology on ad-hoc microphone arrays. Although our method is gen- erative, it does not synthesize harmful speeches that are not initially presented in the speech mixture. Nonetheless, the capability to isolate individual voices may pose privacy risks, which need careful regulations. We believe no other concerns require specific emphasis at this point.
One way to incorporate prior to use a diffiusion model (Ho et al., 2020; Karras al., 2022: Song al., 2021; Sohl- Dickstein et al., 2015). Score-based diffusion model learns
9

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
References
and baseline recipe for multi-channel source separation and recognition, 2019b. URLhttts://arxxte abs 0tt//191.11193
Allen, J. B. and Berkley, A. Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America, 65(4):943-950, 1979.
Drude, L.,
Heymann, J., and Hacb-Umbach, R. Unsuper- vised training of neural mask-based beamforming. arXiv preprint arXiv:1904.0 01578, 2019c.
Amari, S.-i., Cichocki, A., and Yang, A new learning algorithm for blind signal separation. Advances in neural information processing systems, 8, 1995.
Gannot, S., Vincent, E., Markovich-Gonn S., and Oze- rov, A. A consolidated perspective multimicrophone speech enhancement and source separation. IEEE/ACM Transactions Audio, Speech, and Language Process- ing, 25(4):692-730, 2017. doi: 10.1109/TASLP22016 2647702.
Aralikatti, R., Boeddeker, C., Wichern, G., Subramanian, A., and Le Roux, J. Reverberation as supervision for speech separation. In IEFE International Conference on Acousties, Speech and Signal Processing, pp. 1-5, 2023.
Bai, W., Chen, S., Chen, W., and Sun, H. Blind inversion using latent diffusion priors, 2024. URL https:// arxiv. org/ /olpo///400.000000
Han, C., Wilson, K., Wisdom, S., and Hershey, J. R. Un- supervised multi-channel separation and adaptation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 721-725. IEEE, 2024.
Bando, Y., Sekiguchi, K., Masuyama, Y., Nugraha, A. A., Fontaine, M., and Yoshii, K. Neural full-rank spatial covariance analysis for blind source separation. IEEE Signal Processing Letlers, 28: 1870-174,,,, 2021.
Hershey, J. R., Chen, Z., Le Roux, J., and Watanabe, S. Deep clustering: Discriminative embeddings for segmentation and separation. 2016 IEEE international conference on acousties, speech and signal processing (ICASSP). pp. 31-35. IEEE, 2016.
Bando, Y., Aizawa, T., Itoyama, K., and Nakadai, K. Weakly-supervised neural full-rank spatial covariance analysis for a front-end system of distant speech recogni- tion. In Interspeech, pp. 3824-382222 2022.
Hiroe, A. Solution of permutation problem in frequency do- main ica, using multivariate probability density functions. In Rosca, J., Erdogmus, D., Príncipe, J. C., and Haykin, S. (eds.), Independent Component Analysis and Blind Sig- nal Separation, pp. 601-608, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-32631-1
Boeddeker, C. Example of mixture model. https:
//github.u. com/ httpp//bbbbbbbbb/ssseeee. examples 2019.
ipynb,
Cherry, E. C. Some experiments on the recognition of speech, with one and with two cars. The Journal of the acoustical society of America, 25(5)9975-979, 1953.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. arXiv preprint arxiv:2006. 11399, 2020.
Hyvärinen, A., Hurri, J., Hoyer, P. O., Hyvärinen, A., Hurri, J., and Hoyer, P. O. Independent component analysis. Springer, 2009.
Chung, H., Kim, J., Kim, S., and Ye, C. Parallel diffusion models of operator and image for blind inverse problems. In Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, pp. 6059-6069, 2023a.
Ito, N., Araki, S., and Nakatani, T. Complex angular central gaussian mixture model for directional statistics in mask- based microphone array signal processing. In 2016 24th European Signul Processing Conference (EUSIPCO), pp. 1153-1157, 2016. doi: 10.1109EUSIPCO.2016 7760429.
Chung. H., Kim, J., Mccann, M. T., Klasky, M. L., and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In International Conference Learning Representations, 2023b. URL https: ///p/////aee //pppreeeiie www./et//ooooe id=0nD id=Onn92GAATOO 92GAGTOK.
Jayaram, V. and Thickstun, J. Source separation with deep generative priors. In International Conference on Ma- chine Learning, pp. 4724-4355. PMLR, 2020.
Drude, L., Hasenklever, D., and Haeb-Umbach, R. Unsuper- vised training of deep clustering model for multichannel blind source separation. In ICASSP 2019-2019 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 695-699, IEFE, 2019a.
Kalkhorani, V. A. and Wang, D. Tf-crossnet: Lever- aging global, cross-band, narrow-band, and positional encoding for single- and multi-channel speaker sep- aration. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:4999-5000 5009, 2024. doi: 10.1109/AALL.22244444222
Drude, L.. Heitkaemper, J., Boeddeker, C., and Haeb- Umbach, R. Sms-wsj: Database, performance measures,
10

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Karchkhadze, T., Izadi, M. R., and Zhang, S. Improving source extraction with diffusion and consistency models. arXiv preprint arXiv:2412.06665, 2024.
Twelfth International Conference on Learning Represen- lations, 2024. URL https: opwnreviewweeeeeeooeoeeeee net/ forum -dm.mdd222nn22xx
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of iaffusion-basse generative models. In Proc. NeurlPS, 2022.
McDermott, J. H. The cocktail party problem. Current Biology, 19(22):R1044-8)
2009.
Moliner, E., Lehtinen, J., and Välimäki, V. Solving audio
Kim, T., Eltoft, T., and Lee, T.-W. Independent vector analysis: An extension of ica to multivariate components. In International conference on independent component analysis and signal separation, pp. 165-172 Springer, 2006.
inverse problems with diffusion model. In ICASSP 2023 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1 1-5, 2023. doi: 10. 10/1CCASSP99957.200000 202.7999777...0 10095637.
Moliner, E., Elvander, F., and Välimäki, V. Blind au- dio bandwidth extension: A diffusion-based zero-shot approach. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:5092-5105, 2024a. doi: 10. N0.1100TAALLP222433377555
Kinoshita, K., Delcroix, M., Yoshioka, T., Nakatani, T., Habets, E., Haeb-Umback R., Leutnant, V., Sehr, A., Kellermann, W., Maas, R., Gannot, S., and Raj, B. The reverb challenge: A common evaluation framework for dereverberation and recognition reverberant speech. In 2013 IEEE Workshop on Applications of Signal Pro- cessing to Audio and Acoustics, pp. 1-4, 2013. doi: 10.1 A.L10//AASPPAA2213..700122
Moliner, E., Lemercier, J.-M., Welker, S., Gerkmann, T., and Välimäki, V. Buddy: Single-channel blind un- supervised dereverberation with diffusion models. In 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 120-124, 2024b. doi: 10. 110/////C/11133//000000 2024. 10694254.
Ko, T., Peddinti, V., Povey, D., Seltzer, M. L., and Khu- danpur, S. A study data augmentation of reverberant speech for robust speech recognition. In 2017 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), pp. 5220-5224, 2017. doi: 10.1000ICASSPP2217..9933
Moliner, E., Svento, M., Wright, A., Juvela, L., Rajmic, P., and Välimäki, V. Unsupervised estimation of nonlinear audio effects: Comparing diffusiio-bbase and adversar- wwwaiivvvorr........
ial approaches, 2025. URL
abs 0obs/2504.0472
Laroche, C., Almansa, A., and Coupete, E. Fast diffu- sion em: a diffusion model for blind inverse problems with application deconvolution. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pp. 5271-5281, 2024.
Nakashima, T., Scheibler, R., Togami, M., and Ono, N. Joint dereverberation and separation with iterative source steering. ICASSP 2021 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 216-220, 2021. URL https: api semant.cccholle org /ct////or:D:2111111111
Li, A., Yu, G., Xu, Z., Fan, C., Li, X., and Zheng, C. Tabe: Decoupling spatial and spectral processing with taylor's unfolding method in the beamspace domain for multi- channel speech enhancement. Information Fusion, 101: 101976, 2024.
Pandey, A. and Wang, D. On cross-corpus general- ization of deep learning based speech enhancement. IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, 28:2489-2499 2020. URL https: //api ser.d..emanticcccee 211068769.
Luo, Y., Chen, Z., Mesgarani, N., and Yoshioka, T. End- to-end microphone permutation and number invariant multi-channn speech separation. ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 6394-6398. IEEE, 2020.
orq/CorpuIID:
Quan. C. and Li, X. Spatialnet: Extensively learning spatial information for multichannel joint speech sep- aration, denoising and dereverberation, 2023. URL https:/ //aaaxivvvrrgg./.aa /ttps///237.5555
Lutati, S., Nachmani, E., and Wolf, L. Separate and dif- fuse: Using pretrained diffusion model for better source separation. In The Tweifth International Conference on Learning Representations.
Rickard, S.
The DUET Blind Source Separution Al-
gorithm. pp. 217-241. Springer Netherlands, Dor- drecht, 2007. ISBN 978-1 1-4020-6479-1, doi: 10.1007/
Mariani, G., Tallini, I., Postolache, E., Mancusi, M., Cosmo, L., and Rodolà, E. Multi-source diffusion models for simultaneous music generation and separation. In The
978-1-4000-6479--33 URL https: http:/ooooorrgg/
1007/978-1-4020- -40---442------1---

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Rix, A., Beerends, J., Hollier, M., and Hekstra, A. Percep- tual evaluation of speech quality (pesq)-a method for speech quality assessment of telephone networks and codecs. In 2001 IEEE International Conference on Acous- fics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), volume 2. pp. 749-752 vol.2, 2001. doi: 10.1 N0.100//CASPP.2001 24411220.
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 686-690. TEEE, 2022.
Bohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 1256-2265 PMLR, 2015.
Song, J., Vahdat, A., Mardani. M., and Kautz, J. Pseudoinverse-guidee diffusion models for inverse prob- lems. In International Conference on Learning Represen- lations, 2023. URL https: /opnneveeeeeeeeeee net/ forum? -d_____MAMM_KMKK
Roux, J. L., Wisdom, S., Erdogan, H., and Hershey, R. Sdr -half-bkeed or well done? ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 626-630, 630, 2018. URL https: / api wem.eesiccatciccho.ooooo TTT//Co-puSTD:5346655
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative mod- eling through stochastic differential equations. In In- ternational Conference on Learning Representations, 2021. URL https:: pttttps://openeviiiii /twp//pppwpeeeeeeeeeeeeee net// forum? id-PXTIG12RRES.
Saijo, K. and Ogawa, T. Self-remixing: Unsupervised speech separation via separation and remixing. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1-5. IEEE, 2023.
Subakan, Y. C. and Smaragdis, P. Generative adversar- ial source separation. In 2018 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), pp. 26-30. IEEE, 2018
Saijo, K., Wichern, G., Germain, F. G., Pan, Z., and Le Roux, Enhanced reverberation as supervision for unsupervised speech separation. Interspeech 2024, pp. 607-611, 2024. doi: 10.214477/mtersseechh 2024-1121..2211111.
Švento, M., Moliner, E., Juvela, L., Wright, A., and Välimäki, V. Estimation and restoration of unknown nonlinear distortion using diffusion. arXiv preprint arXiv:2501.05959, 2025a.
Sawada, H., Araki, S., and Makino, S. Underdetermined convolutive blind source separation via frequency bin- wise clustering and permutation alignment. IEEE Trans- actions Audio, Speech, and Language Processing, 19 (3):516-522 527, 2011. doi: 10.1 101110//SL.2001.25551111
Švento, M., Moliner, E., Juvela, L., Wright, A., and Välimäki, V. Estimation and restoration of unknown nonlinear distortion using diffusion. arXiv preprint arXiv:2501.05959, 2025b.
Sawada, H., Ono, N., Kameoka, H., Kitamura, D., and Saruwatari, H. A review of blind source separation meth- ods: two converging routes to ilrma originating from ica and nmf. APSIPA Transactions on Signal and Information Processing, 8:e12, 2019.
Taal, C. H., Hendriks, R. C., Heusdens, R., and Jensen, J. An algorithm for intelligibility prediction of time-frequency weighted noisy speech. IEEE Transactions on Audio, Speech, and Language Processing, 19(7):2125-2) 2011. doi: 10.1 10.1199/AASL.221122 148881
Scheibler, R. and Saijo, K. torchiva: Tools for independent vector analysis in pytorch. bettp://ggitiin com/ 2022.
takufaku/torchte
Taherian, H., Eskimez, S. E., Yoshioka, T., Wang, H., Chen, Z. and Huang, X. One model to enhance them all: array geometry agnostic multi-channel personalized speech en- hancement. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 271-275. IEEE, 2022.
Scheibler, R., Ji, Y., Chung, S.-W., Byun, J., Choe, S., and Choi, M.-S. Diffusion-based generative speech source separation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, IEEE, 2023.
Seetharaman, P., Wichern, G., Le Roux, J., and Pardo, B. Bootstrapping Lingle-chanee source separation via unsu- pervised spatial clustering on stereo mixtures. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 356-360. IEEE, 2019.
Taherian, H., Pandey, A., Wong, D., Xu, B., and Wang, D. Multi-input multi-output complex spectral mapping for speaker separation. ISCA, 2023.
Taherian, H., Pandey, A., Wong, D., Xu, B., and Wang, D. Leveraging sound localization to improve continu- ous speaker separation. In ICASSP 2024 2024 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), pp. 621-625, 2024. doi: 10. N0.1109//CASSP444RR. 444444484445222224444444444444
Sivaraman, A., Wisdom, S., Erdogan, H., and Hershey, J. R. Adapting speech separation to real-world meetings us- ing mixture invariant training. In ICASSP 2022-2022
12

---

ArrayDPS; Unsupervised Blind Speech Separation with a Diffusion Prior
Thuillier, E., Lemercier, J.-M., Moliner, E., Gerkmann, T., and Välimäki,, V. Hrtf estimation using score- based prior. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
training mixtures. In Thirty-seventh Conference on Neu- ral Information Processing Systems, 2023. URL https: / //opprreeieee www.:/etttoodd-teeeeeeeeee
Wang. Z.-Q.. Le Roux, J., and Hershey, J. R. Multi- channel deep clustering: Discriminative spectral and spa- tial embeddings for indgeeeeinde speech separa- tion. In 2018 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), pp. 1-5, 2018. doi: 10.1 1AL109/CAASSP22188866....
Togami, M., Masuyama, Y., Komatsu, T., and Nakagome, Y. Unsupervised training for deep speech source separa- tion with kullback-leibler divergence based probabilistic loss function. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 56-60. IEEE, 2020.
Wang, Z.-Q.. Wichern, G., and Le Roux, J. Convolutive prediction for reverberant speech separation. In 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics ( (WASPAA), pp. 56-60, 2021a. doi: 10.1 1..109/AAAAAS5581 w.2.:A555881222...0 2022.0632666999
Tran Vu, D. H. and Hlcb-Umbachh R. Blind speech sepa- ration employing directional statistics in an expectation maximization framework. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 241-244, 2010, doi: 10. 10.109/CAASP..0005555944
Wang, Z.-Q., Wichern, G., and Roux, J. L. Convolutive prediction for monaural speech dereverberation and noisy- reverberant speaker separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3476- 3490, 2021b. doi: 10. 10.1109TAASLP.2222.112222
Tzinis, E., Venkataramani, S., and Smaragdis, P. Unsu- pervised deep clustering for source separation: Direct learning from mixtures using spatial information. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 81-85. IEEE, 2019.
Wang, Z.-Q., Cornell, S., Choi, S., Lee, Y., Kim, B.-Y., and Watanabe, Tf-gridnet: Integrating full- and sub-band modeling for speech separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:3221- 3236, 2023. doi: 10. N0.1109/TASSL.222333304
Tzinis, E., Wisdom, S., Jansen, A., Hershey, S., Remez, T., Ellis, D. P., and Hershey, J. R. Into the wild with audioscope: Unsupervised vistul separation of on- screen sounds. arXiv preprint 2rXiv:201 1.00 2020.
Wisdom, S., Tzinis, E., Erdogan, H., Weiss, R., Wilson, K., and Hershey. J. Unsupervised sound separation using mix- ture invariant training. Advances in neural information processing systems, 33:3846-333 0113846-857,, 2020.
Tzinis, E., Adi, Y., Ithapu, V. K., Xu, B., Smaragdis, P., and
Kumar, A. Remixit: Continual self-training of speech
enhancement models via bootstrapped remixing. IEEE Journal of Selected Topics in Signal Processing, 16(6): 1329-1341, 2022a.
Xu, A. and Choudhury, R. R. Learning to separate voices by spatial regions. In International Conference on Machine Learning, pp. 24539- 24549.. PMLR, 2022.
Xu, Z., Aroudi, A., Tan, K., Pandey, A., Lee, J.-S., Xu, B., and Nesta, F. Fovnet: Configurable field-of-veew speech enhancement with low computation and distortion for smart glasses. In Interspeech 2024, pp. 3350-3354, 2024. doi: 10. 10.21447//neesppecch222444 2024-2214.....2
Tzinis, E., Wisdom, S., Remez, T., and Hershey, J. R. Au- dioscopev2: Audio-visual attention architectures for cali- brated open-domain on-screen sound separation. In Eu- ropean Conference on Computer Vision, pp. 368-385. Springer, 2022b.
Yoshioka, T., Wang, X., Wang, D., Tang, M., Zhu, Z., Chen, Z., and Kanda, N. Vararray:
Vincent, E., Gribonval, R., and Fevotte, C. Performance measurement in blind audio source separation. IEEE Transactions on Audio, Speech, and Language Process- ing, 14(4):1422-1469, 2006. doi: 10.1109/TS..2005. 858005.
continuous speech separation. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6027-6031. IEEF, 2022.
Yu, C.-Y,, Postolache, E., Rodolå, E., and Fazekas, G. Zero- shot duet singing voices separation with diffusion models. arXiv preprint arXiv:2311.0743 2023.
Wang, D. and Chen, J. Supervised speech separation based on deep learning: An overview. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(10): 1702-1726, 2018. doi: 10. 10.1099TASLP.220882422259
Yu, D.,, Kolbæk, M.. Tan, Z.-H., and Jensen, J. Permu- tation invariant training of deep models for speaker- independent multi-talker speech separation. In 2017 IEEE International Conference on Acoustics, Speech and
Wang, Z.QQ. and Watanabe, S. UNSSOR: Unsupervised
neural speech separation by leveraging over-determined
13

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Signal Processing (ICASSP), pp. 241-245, 2017. doi:
10.1 00.100/CAASP2221..952555
Zen, H., Dang, V., Clark, R., Zhang, Y. Weiss, R. J., Jia, Y., Chen, Z., and Wu, Y. Libritts: corpus derived from librispeech for text-to-speech. In Interspeech 2019, pp. 1526-1530, 2019, doi: 10.21457/mierrseechh201999 9011
Zhang. W., Shi, J., Li, C., Watanabe, S., and Qian, Y. Clos- ing the gap between time-domain multi-channe speech enhancement on real and simulation conditions. In 2021 IEEE Workshop Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 146-150, 2021. doi: 10.1 1091/ASSPAA52558 9632770..
14

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
We organize the appendix as follows:
Appendix A provides a background on score-based diffusion and diffusion posterior sampling (DPS).
Appendix B provides proposed posterior score derivation, analysis, and relation to the classic EM algorithm.
Appendix C presents Algorithm and Algorithm 2 in detail, and provides algorithm configurations.
Appendix D explains diffusion prior training configuration and model architecture.
Appendix F discusses all the baseline models in detail.
Appendix E illustrates the SMS-WSJ and Spatialized WSJ0-2Mix datasets for evaluation.
Appendix G shows ablation experiments for FCP method and explains why we use the virtual source signal model.
Appendix H reports the result of 2-channel and 3-channel Spatialized WSJO-2Mix dataset.
Appendix reports the result of 6-channel SMS-WP WSJ dataset.
Appendix reports the 3-speaker separation results on the 4-channel Spatialized WSJ0-3Mix dataset.
Appendix K shows ArrayDSS's sensitivity and robustness to hyperparameters.
Appendix L shows the visualization of ArrayDPS separated virtual sources, FCP estimated RIRs, and ArrayDPS separated reference channel sources.
A. Score-based Diffusion and Diffusion Posterior Sampling
Score Diffusion Model: Diffusion- generative models aim to sample from a data distribution Pdat (s), by starting from a noise sample and then gradually denoising it (Song et al., 2021; Karras et al., 2022: Ho et al., 2020). Score-based diffusion model (Song et al., 2021; Karras et al., 2022) can also formulate the diffusion denoising process with a probabilistic flow ODE. Following EDM (Karras al., 2022), with the specific diffusion noise schedule (r)) where is the diffusion time, then the probabilistic flow ODE is defined by:
ds = -T) log (dd
(21)
where p(s") (s. (T)T). This probabilistic flow ODE allows transformation from 8mmn (noise) to s (data), using an ODE solver like Fuler's method. During training, denoiser Do(s" a(()) is learned to denoise S with the following denoising loss:
ErEss (0. Da(s + d(r)) - s|[?
(22)
After training, the score function V.- logg(ps)) is approximated by So(s", (T)) =: Dots" _(())= using the Tweedie's (²
Formula. During sampling, sua first sampled from (N00 0² (Tmma) and then an ODE solver is used integrate through Eq.21 from Tees data sample y Tmie
Pdam-
This diffusion model further allows a universal framework to solve the general inverse problem. which is known for diffusion posterior sampling (Chung et al., 2023b; Song et al., 2023). Consider a clean signal (e.g., image speech) is distorted by distortion function A(-), which results in a distorted signal A(s) + e, To recover the original clean signal s, diffusion posterior sampling proposes sample from p(s(2) by using the probabilistic flow ODE similar Eq. 21 except using the posterior score:
Diffusion Posterior Sampling:
ds" = Vr log ((s))
(23)
To approximate the posterior score V.- ((s (z)) it's decomposed into prior score and likelihood score with Bayes' theorem:
Ver log (z) = V logp(s") + V.r logp(x|s")
(24)
where v.- log p(s") ) is estimated by So (T)), and in DPS (Chung et al., 2023b), the second term V.- log
ss)
empirically approximated by:
Ver log]((ss' )). where (s("") )- = Do(s", (())
(25)
[> A(Do(s", (
(26)
The E(7) is a parameter to control the amount of likelihood guidance for each step, which theoretically relates to the measurement noise variance. Note that this solution assumes that the distortion function A(-) is known in advance.
15

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
B. Posterior Score Approximation
This section first gives derivation of the proposed posterior score approximation as in Sec. 3.3 Eq. 18. Then to understand why this makes sense, we give thorough analysis of our upproximation in Sec. B.2, where also prove FCP maximum likelihood estimator. Finally, in Sec. B.3, we connect our likelihood score approximation step with the classic expectation maximization algorithm.
B.1. Derivation
This section gives a full detailed derivation of our approximation of the posterior score as proposed in Sec. 3.3. Basically, goal is to approximate V=Ix.. log p(s (21:c) that's needed for sampling. First, similar to Eq. 24, we decompose T logp((f.g((ree) with the Bayes' theorem:
i.x logp(si = V
six log (sg + V "ixx logp( s1.k)
(27)
V logp(s{) V V logp(snc((1..)
(28)
Now notice that in the first part of Eq. 28, all the summands V sg log (prior score) can be approximated by the score model So (8% (T)) trained on single-speaker speech. Thus, we try to approximate the second part of Eq. 28 (likelihood score):
V L logp(<1:c(5.k)
V Lix logp(I.(si.x)
(29)
V f log ((.
where sQ(sT) (
(30)
Eq. 29 is correct because assume that for the mixtures, different channels' mixtures conditionally independent given all the sources, which is our array-agnosti assumption. Eq. 29 to Eq. 30 is based on the likelihood approximation in DPS, as shown before in Eq. 25. However, in Eq. 30, (= clean sources are single-channel virtual sources, while the mixtures are multi-channel. To build the connection, we transfer the STFT domain and then estimate the relative RIRs transform from virtual sources to real channel source images.
8% (sx))
)) is still not tractable because the estimated
First, follow the notation in Sec. 3.2, denote X, the STFT of x. and denote Se as the STFT of ( Then the FCP algorithm defined in Sec. 3.2 Eq. 12 and Eq. 15 to estimate the relative RIRs which allows the transformation from virtual source estimates to all-channel source images:
Go FCP(X,, $2), Sa,e Gl 52 K (1,2,...,K), CE (1,2, C}
(31)
Now given the estimated source images Sk,es are able to approximate the likelihood score in Eq. 30 in tractable way:
Vsi S4
& (*k))
(32)
Lix logp(X,
50)
(33)
log p X.
20
50 Gcl
Gomee....
GK
(34)
fix
K
2
= V E(T)
X,
Go (1
ck
50
(35)
k=1
K
= -Vsixx((T)
X-
Sk,
(36)
k=1
~V 2/ {(7)
ISTFT
Sk
(37)
16

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Eq. 32 to Eq. 33 is just STFT transform. Eq. 33 Eq. 34 is an empirical approximation. A simple intuition of this is that Eq. 34 is tractable while Bq. 33 is not. The reason is that the relative RIRs Gb needed for the likelihood be tractable, according to the signal model in Eq. 11. Thus an empirical choice is to estimate the Gb using FCP, as in Eq. 31. Later in next section, we will show that these FCP estimated relative RIRs GI-K 1-10C are exactly maximum likelihood relative RIRs:
GK G G-scr uy Go-r
arg max P(X SK Go G0-te:.. G Gb Ser...G
(38)
We analyze the validity from Eq. 33 to Eq. 34 (likelihood score approximation) in the next subsection where we also show the connection to the classic Expectation Maximization (EM) algorithm. Eq. 34 to Eq. 35 based on the signal model in Eq. 11 with the Gaussian measurement noise. Eq. 35 to Eq. 36 is a simplification using Eq. 31, and Eq. 36 Eq. 37 based on the power preservation of STFT the STFT version of Parserval's 's theorem.
Now can finalize our derivation of the approximated posterior score using Eq. 28, Eq. 30, and Eq. 37, which derives result in Eq. 18 Sec. 3.3:
ai log logp(six(mn.)) Σ
Σ
V six {(T)
te - ISTFT
Sk.e
(39)
B.2. Analysis and Validation
This section complements the derivation above showing the analysis and validation of our posterior score approximation. We first show that the relative RIR filter estimation using FCP is equivalent maximum likelihood filter estimation, which would validate Eq. 38. We then validate the likelihood score approximation from Eq. 33 to Eq. 34. Lastly, we show the likelihood score approximation' 's relationship with the classic Expectation Maximization algorithm.
To show the FCP as a maximum likelihood relative RIR estimator, we first recap the FCP formulation in Eq. 12 and Eq. 13,
G (. $x0,1)
= arg min Σ Go (L. Lf NE(I
1 Xell,S) Go $2
(40)
K C. f) + Ncll.), where Nellff ~ NNO,, o3.). Then when
Theorem B.1.
Assume Xx(l,f) = Σ k=1
(t(, = 20
J. the FCP relative RIR estimator is a maximum likelihood filter estimator:
FCP(X,, S), FCP(( Sk)= = arg max logp(Xx Gb $0 S Sk.
(41)
GB G. GK
Proof. First, let Gb p GK Ge be the maximum likelihood estimator as shown below:
G Gu my GK
arg max Go-
log
GK
Gb 50 $%)
(42)
arg max Nogo
Σ
Go-so #1 50 1)
G
(43)
k=1
arg min
X
Σ
Go
GK
*1 30
(44)
2
Go GK
Eq. 42 Eq. 44 is based on the signal model in the assumption. Then based on the orthogonal principle in estimation theory, independence of the sources, and linearity of the filtering operation, Eq. 44 is equivalent to:
1 X
Glac *1 s01
Go = arg min Gh
vk E 21, K}
(45)
2
which is exactly the FCP objective mentioned in Eq. 40, where 1t(1, f)
26 1 in the assumption.
17

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
With Theorem B.I, it is also obvious that FCP is equivalent to the maximum posterior solution of the relative RIR because do not assume any prior of the relative RIR filter. Note that empirically in FCP, the weight Xt f) is set to - - Ee [x,(1,)) + max 1J
E [x,(1, for better regularization.
Now with Theorem B.I relating FCP with filter likelihood, we further analyze the correctness of the likelihood score approximation from Eq. 33 to Eq. 34 in the derivation section B.1. From Eq. 33 to Eq. 34, we have the FCP estimated relative RiRs so that the likelihood is tractable:
V L log (( .... 5k) = V logp(X.(5) 1:... 50 G Goeer my Go-re)
(46)
Now we find the assumption that would make the approximation into equality:
p P(X.|Sp
S0 ....,, 5kk
pCG .. Go-c)p(( (x. ..., 50 Gb-rer... ..y (0- dG
(47)
Gf.....G...
{1 if Go G G0+c for all k
5k. 0o ... ( if (X Sk. G Go Go
else
(48)
We can see that the approximation becomes equality under the assumption that p (x..55t S% Gi Gf-sc) is a delta function of all the relative RIRs G Gl-set..., G where the function has non-zeros values only when the relative RIRs are 0 which are also the maximum likelihood estimators mentioned before. This assumption makes sense when there is measurement noise, where slight error in the relative RIR would cause likelihood of O. However, of course, when there is relatively larger measurement noise, the assumption is no longer true, but would still work empirically as shown in our SMS-WSS ASS-WSJ separation result in Table 1.
the FCP estimated relative RIRs G G0- G
B.3. Likelihood Approximation and Expectation Maximization
While we analyze the correctness of our posterior score approximation in the previous section, we also find that the most challenging step (likelihood score upproximation Eq. 46) has a strong connection with the classic Expectation Maximization algorithm. Note that in EM algorithm, when the likelihood p(..l$p..., relative RIRs G
is intractable because of the unobserved G ..., G. the Expectation (E) step and the Maximization (M) step are iterated:
E-Step: calculate u (G ".. GR + p(G ... GK ( S6....
(49)
M-Step: update $9
20 SK
f- arg max E B GK [log p(X.Ss 56. Go GG
(50)
$g.....3
30
The E step updates the distribution & (G)-re...., GK of the unobserved relative RIRs to be the distribution of the relative
RIRs given the current source estimates 59 . S and the mixture X Then using the E step updated distribution Ge).). the M step maximizes the expectation of log likelihood by update the source estimates Sp.... 8g. Then this procedure is iterated until convergence, which is called the Expectation Maximization algorithm.
In our case, we only want to estimate the current log likelihood without updating the sources estimate SP.... SR so our log likelihood approximation only contains one step and then calculate the expected log likelihood as in the M step (Eq.50). However, The E step is also not tractable in our case, so we just approximate it with:
if G Gb for all k
(G
G
(51)
0 else
G2.7 where G/
G0 m. G0
GK
arg max Ck
log log G Sq..., 5%
(52)
argmax Gk
log Dogp(X.GG)) 00bay ". Go S1, s,,.., Skk
GK
2u 60
(53)
= S9), FCP(X.. 59), FCP(X 5%)
(54)
We set the distribution d (G) GK in the E step to be deterministic at Gl GK Go which are both the maximum likelihood filters and maximum posterior filters because no priors are assumed for the filters. Then as mentioned in Sec. B.2
18

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Theorem B.1, C m.y GK can be directly estimated by the FCP method. Then after the E step, we can calculate the
expected log likelihood in M step by only calculating log p(X,]Ss,, -m KK G 00 ..y GK 00+c) because y is defined to be deterministic distribution.
C. Algorithm Details
This section provides all the details of the two algorithms in Sec. 3.5, including ArrayDPS as in Algorithm I and posterior score approximation in Algorithm 2.
C.1. ArrayDPS Algorithm
This section discusses the ArrayDPS Algorithm in detail.
IVA Initialization: From lines 1-6 in Algorithm 1, IVA first separates K speech source images in the reference channel (channel 1). Then these IVA-separated sources are used as source estimates calculate the relative RIR using FCP. Note that in lines 4-6, the relative RIR 6 I-c is calculated for all channels. These IVA-initialized relative RIRs will be later used for posterior score approximation as in Algorithm 2, which will be discussed later. Then line 7-9 initializes the starting diffusion noise with the IVA separated sources, in the reference channel (channel 1). Although we are trying to output a virtual source, we still initialize it to be the reference channel's IVA outputs.
Lines 10-24 Algorithm show the discrete sampler that integrates through the separation ODE in Eq. starting from the diffusion noise initialized in line This sampler is proposed in EDM (Karras al., 2022), with the parameters oi (o....... N-1)- : Ve (o...... 1): Smoiee ) and inputs {I K] F1 is the multi-channel mixture input and K is the number of sources to separate. N is the number of diffusion denoising steps. (o....... is the noise schedule for each
Stochastic Sampler:
diffusion denoising step (i.e., amount of noise to remove at each step), where ON = 0, oo = a (Tmax)) CN-1 a (Tmii))
Then all other steps' schedule follows:
oieN = (T((mxx))
N 1 (a(rmin) ( - o (Tmm))
(55)
p is parameter that controls how the noise level per step changes from (Tmaz) (Tmin)). ), e.g., p >> 1 means the noise level per step will decrease more rapidly increases. Note that in our ODE settings in Scc.3, (T.) Ti 0x. Thee (I....N-1I) are parameters to control the amount of stochasticity added in the beginning of each step, as in lines 11-14. In EDM (Karras et al., 2022), "iE(1.....NI1) is set to be:
= min
Sum
1)) if o E [Smins Smax] N
(56)
otherwise
where Sehum controls the amount of stochasticity and SminnSSmnn set the steps for adding stochasticity. Smoise in line 12 also controls the amount of stochasticity within step E [Smin: Smar]. In general, the sampler first has a usual 1 order Euler step to update the sources (1 (16-18), and then the newly updated source is used to calculate 2nd order correction step (lines 20-22). More details of the sampler can be checked in the original EDM paper (Karras et al., 2022).
Post FCP Filtering: Note the sampler mentioned above in Algorithm 3.5 is separating virtual sources (line 24). To get the final separated source images reference channel 1, post-filtering is needed. Lines 26-29 use the FCP algorithm to estimate the relative RIR that can transform the virtual sources into the reference channel. Then after convolutional filtering (line 28), the final reference channel-separatee source images returned as outputs (line 30).
C.2. Posterior Score Approximation Algorithm
In Algorithm 1, each update step needs to calculate the posterior score, which uses the GET_SCORE function in Algorithm 2. The function GET_SCORE(s") FT.C. GV.IVA et i) aims approximate the posterior score V log p( ( ( at sampling step i, using our propose approximation formulation in Sec. 3.3, but with some empirical modifications. The function needs a few pre-defined parameters ( (Do (ie(O...., Niet, Nig, Ei (27)) Do is the diffusion denoiser mentioned in Sec. 3.3. (iE(o....,N) is the sampler's 's noise schedule as mentioned in the stochastic sampler paragraph.
Net is a sampler step threshold such that when S Nect there is an empirical reference-chamie guidance term that aims to match the sum of estimated virtual sources Σ Σ &g to the reference channel (channel 1) mixture æ1. This shown in line
19

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
18 in Algorithm. 2. We found that this is important to the algorithm's 's stability. Nig is also sampler step threshold for the filtering guidance of IVA initialized relative RIR. As shown in lines 7-14, when S Nig. instead of using FCP to estimate the relative RIR, the IVA initialized relative RIR G c is directly used. We found that this helps a lot for sampling stability and performance gain. The intuition is that in the first Ng steps, the source estimate would be extremely blurred and so as the relative RIR estimates, while the IVA initialized relative RIR is much better choice. Note that GV.IVA c are actually relative RIRs from reference channel I to all channels, but our estimated sources are all virtual sources. We clarify here that this is on purpose, which means in the first Ng steps, the virtual source estimation would be close to the reference channel, and then the constraint is relaxed in later steps.
(1) and (2(T) are the weights for the mixture likelihood guidance in lines 16 and 18 in Algorithm. 2. Theoretically, (I(T)= . where o is the white noise variance defined in Eq. 2, and {2(T) = 0, because the term in line 18 is empirical. However, for sampling stability, we resort to the common practice used in previous audio posterior sampling work (Moliner et al., 2023; 2024a;b), where we set & (7) and (T) to the following:
EVT
(T)
(57)
C
2
V
Σ
dic
-
E
8k
0=1
2 2
EVT
E2(T)
(58)
T
V xf
[1
All the variables in Eq. 57 and Eq. 58 directly refer to the same variables in lines 16 and 18 in Algo. 2. I is the length of the audio samples. Ç is just single scalar parameter to tune the mixture likelihood guidance. Note that (1 and (2 directly depend on the calculated gradient term so the the notations (T) and (2( are not accurate. However, we stick to it because the notation is simpler and it is also used in previous papers (Moliner et al., 2023; 2024a).
Overall, in Algorithm 2, the diffusion denoiser first denoises the noisy virtual sources (lines 3), and then the prior score calculated using Tweedie's 's formula (line 6). Then depending on the current sample step i, the algorithm either chooses the IVA initialized relative RIR (line 12) or directly estimates the relative RIR (line 10) using FCP. The relative RIR is then used to filter the virtual source estimates to all real channels (channel I to C) (line 14), which allows the calculation of the gradient likelihood guidance (line 16) and the reference channel guidance (19). Finally, the prior score and the guidance terms are added together to output the approximated posterior score (line 21).
C.3. Algorithm Configurations
Here we show our ArrayDPS configurations. For the default configuration as in row 2a (ArrayDPS-A) in Table 1, we
set N = 400 and Smoise = as in Algorithm 1, ao = Tmax 0.8, ON = Tiin = le - 6, p = 10 as in Eq. 55, Smin = O, Smax = 50, and Sehum = 30 as in Eq 56, 5 = 2, Nief = 200, Ng = 100, and A = 1.3 as in Algorithm 2. The diffusion prior trained on anechoic speech is used. For the FCP algorithm, we use an FFT size of 512 (64 ms), hop size of 64 (8 ms), square root Hanning window, and e 0.001 as in Eq. 14. For the IVA initialization, we use the open-source torchiva toolkit (Scheibler & Saijo, 2022), and we FFT size of 2048 (256 ms), hop size 256 (32 ms), Gaussian Prior, and 100 iterations.
For ArrayDPS-B B in row 2b of Table 1, the diffusion model is trained on reverberant speech, as discussed in Sec. 4.2 and Appendix D. For ArrayDPS-C in row 2c of Table 1, the reference channel guidance in Algorithm 2 is removed, which means ==0 O. For ArrayDSS-D and ArrayDPS-E in row 2d-2e of Table 1, the IVA initialization is removed, which means Ng = O and no IVA is needed. Moreover, in the case of no IVA initialization, we set op = Tmax = 2 and & = 6 since we found this set of parameters perform better when tuning on validation sets. For ArrayDPS-E, the reference-channel guidance is also removed, so Ny 0 in this case.
D. Diffusion Training Details
As mentioned in Sec.4.2, the diffusion prior is trained on the train-clean- 100 and train-clean-38 subset of LibriTTS dataset (Zen et al., 2019). This subset contains about 460 hours clean speech with 1000+ speakers. However, this training set is primarily reverberation-fret or anechoic. Since the virtual source could be reverberant, we also train a second diffusion
20

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
model on reverberant speech. The reverberant speech dataset is the same subset of LibriTTS, but each speech utterance is convolved with randomly selected room impulse response (RIR) to synthesize reverberant speech. The randomly selected RIRs are from the SLR26 and SLR28 dataset (Ko et al., 2017), which contains 3,076 real and about 115,000 synthetic RIRs.
For the diffusion denoising architecture, we use the waveform domain U-Net as MSDM (Mariani et al., 2024), implemented in audio-diffusion wwwffusion-yyttrre//vaae
The U-Net takes audio waveform as inputs, and also outputs audio waveforms. The U-Net consists of encoder, bottleneck, and decoder. The encoder contains 6 layers of 1-D convolutional ResNet blocks, where the last three layers also contain multi-head slef-attention blocks following the ResNet block. The input channel is I because we are modeling single channel speech. The number of output channels for each encoder layer is [256, 512, 1024, 1024, 1024, 1024], and the number of downsampling factors for each layer is [4, 4, 4, 2, 2, 2]. For self attention blocks, we use 8 attention heads where each is 128 dimensional. The bottleneck also contains an attention block, with one ResNet block before the attention block and one ResNet block after the attention block. The decoder is then reverse symmetric to the encoder, with output channel size I to output a same size waveform.
For diffusion training configurations, we use the EDM (Karras et al., 2022) training framework following the training recipe in CQT-Diffusion'. The diffusion denoiser Do (T)) is set to be linear combination of the noisy input and the U-Net fe output:
Do(s", (TT)) S Coula(T)) fo(cms" Croise)
(59)
(60)
.where Coiip(),, (o), Cin Ci)(or, Smnise()). are defined as:
Cskip (o) 02 + haam
(our(o)
σ Odta
Gi cin(o)=
1 Vo2
Caoise (o) In(o)
1
(61)
V + o
+ data
4
data shown above is parameter to set based on the standard deviation of the training dataset. We set this value to 0.057.
Then the diffusion denoising objective is shown
E Errpp.Bsppaa En- (()) s||
(62)
where p. is empirically set to match the diffusion sampling scheduler's density. During training, we set (Tmax) 10.
(TTii)) = le 6, 10, Schum = 5 for the diffusion scheduler. We train on speech samples with 65,536 samples (~~8.2
s) with batsh size 16 and learning rate 0.0001. The learning rate is multiplied by 0.8 every 60,000 training steps. Also exponential moving average (EMA) is used to update the neural network weights during training. The EMA updated weights after training is used for sampling in inference time. We train the model for 840,000 training steps.
E. Evaluation Datasets
In this paper, two datasets are used for evaluation. The first dataset is SMS-WSJ (Drude et al., 2019b) for fixed microphone array evaluation, where all the methods are evaluated on one single microphone array recorded samples. The second dataset is Spatialized WSJO-2Mix ( (Wang et al., 2018) for ad-hoe microphone array evaluation, where methods are evaluated on multiple unknown microphone arrays.
E.1. SSS-WSJ Dataset
SMS-WSJ (Drude et al., 2019b) is a commonly used reverberant speech separation corpus (Wang & Watanabe, 2023; Taherian et al., 2024; Saijo et al., 2024; Kalkhorani & Wang, 2024). The dataset is simulated from WSJO and WSJI datasets. The dataset is for multi-channel 2-speaker separation in reverberant conditions for a fixed simulated microphone array. The simulated microphone array a circular array with six microphones, uniformly on circle with 10 cm of radius. For each sample, the shoebox room is randomly sampled, with the length and width uniformly sampled from [7.6, 8.4] and [5.6, 6.4] m, respectively. The array center position is randomly sampled from [3.6, 4.4) m from the shorter wall and [2.6, 3.4] from the longer wall. Then the array is randomly rotated along all three geometric axes. The mixture contains two speaker sources, which are sampled form WSJO and WSJI datasets. For each speaker, the speaker-to-array distance is uniformly
21

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
sampled from [1.0, 2.0] m, and each speaker position's is azimuth related the array center is also randomly sampled. The room impulse responses are simulated using image-source method (Allen & Berkley, 1979). The sound decay time (T60) is uniformly sampled from [200, 500] ms. Finally, to simulate sensor noise, SNR value is sampled to be [20, 30] dB, and then the white noise is sampled and scaled to satisfy the SNR required. The dataset consists 33,561 (~~87.4 h), 982 (~2..5 h), and 1 1,332 (~3.44 h) train, validation, and test mixtures, respectively, all in 8kHz sampling rate.
E.2. Spatialized WSJ0-2Mix Dataset
Spatialized WSJO-2Mix (Wang et al., 2018) is the spatialized version of the WSJO-2Mix (Hershey et al., 2016) dataset, which is a commonly used speech separation dataset mixed by randomly selecting sources from the WSJO corpus. Multi- channel Deep Clustering (Wang et al., 2018) creates the Spatialized WSJO-2Mix dataset for multi-channel reverberant speech separation. For each dataset sample in WSJO-2Mix, a shoebox room, 8-channel microphone array geometry, source positions, and microphone array center are sampled randomly to spatialize that sample. The room length I, width w, and height are uniformly sampled from [5, 10] m, [5, 10] m, and [3, 4] m, respectively. The RIRs' sound decay time (T60) is sampled from (200, 600] ms. The microphone array center is set to be (1/2 + n1, w/2 + h22 where ii and n2 are uniformly sampled from [-0.2, 0.2] m, and h is uniformly sampled from [1. 2] m. Then the microphone array geometry is randomly sampled. First, the array radius is uniformly sampled from [7.5, 12,5] cm. The first two microphones are sampled on the sphere with the sampled radius, and are symmetric according to the array origin. Two more microphones are randomly sampled inside the sphere while making sure the distances between any two microphones has to be least em. Then 4 more microphones are randomly sampled inside the sphere without any restrictions. In total, eight microphones
randomly sampled. For the speakers, each speaker's location (ar, y, =) is sampled such that 2. The source locations are
1/2 + m1 1.5 s 1/2 + n1 + 1.5, w/2 + n22 1.5 S y I w/2 + m2 + 1.5, 1.5 S sampled such that the source-array distance is at least 0.5 and source-source distance is at least m. The RIR simulation is also using the image source method (Allen & Berkley, 1979). The sample rate is set to 8kHz.
In general, the Spatialized WSJO-22Mix dataset contains 20,000(~30h), 5,000 (~~10h. and 3,000 (~5h) utterances in training, validation, and testing, respectively. Different utterances are using different microphone geometries, allowing training models that can generalize diverse ad-hoc microphone arrays. Also, the direct-to-reverb energy radio of the dataset is 2.5 dB with 3.8 dB of standard deviation.
F. Baseline Methods
As in Row la of Table 1, Spatial clustering (Rickard, 2007; Tran Vu & Haeb-Umbach, 2010; Sawada et al., 2011; Ito et al., 2016) tries to cluster the multi-channel spatial features in STFT domain. Then each STFT bin is assigned to a source based on the clustering, which relies on the assumption that different speech do not overlap in STFT domain, known as W-Disjoint Orthogonality (W-DO) (Rickard, 2007). We use the same spatial clustering configuration as in the baselines of UNSSOR (Wang& & Watanabe, 2023), which uses the open source implementation (Boeddeker, 2019). The FFT size is 1024 (256 ms) and the hop size is 128 (32 ms). The number of sources is set to K + 1 and the lowest energy source is discarded as garbage source.
For the Independent Vector Analysis baseline, similar to the baseline in UNSSOR (Wang & Watanabe, 2023), we use the torchiva toolkit with FFT size 2048 (64 ms) and hop size 256 (32 ms). In UNSSOR, the spherical Laplace prior is used as in Row 1b of Table We also include Gaussian prior because we find it to perform better, as shown in Row 1c of Table 1.
For the UNSSOR baseline trained Spatialized WSJO-2Mix datasets, we use the default configuration where both MC and ISMS loss are used, and / 19 and J O, following the notation in the paper. We use the open source code" to train UNSSOR. We found that UNSSOR training is quite sensitive to the ISMS loss weight, which is in the original paper. Thus we sweep E {0.03, 0.04, 0.05, 0.06, 0.08, 0.3) and report the best result for the 2/3/4-channel of Spatialized WSJO-2Mix dataset. For the SMS-WSI WSJ dataset, we just borrow the result in the original UNSSOR paper and report as UNSSORT i as in Table
Row 5a-5c Table 1 show the supervised TF-GridNet (Wang et al., 2023) with permutation invariant training (PIT) (Yu al., 2017), where TF-GridNet-SMSI is the exact model trained on SMS-WSJ dataset provided as the supervised baseline of UNSSOR, showing superior performance. However, we also need supervised baseline model trained on the Spatialized WSJ0-2Mix dataset. Thus we reproduce a supervised TF-GridNet training pipeline using the same architecture with
22

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
TT-G in UNSSOR, but trained with with SI-SDR(Roux al., 2018) PIT loss for simplicity (we clear about the training details of the TF-GridNet supervised baseline reported in UNSSOR). We use ADAM optimizer and start with le-4 learning rate. We half the learning rate whenever the validation loss do not improve in 3 consecutive epochs. We use batch size 4 and each training segment is 4 seconds long. As in Row 5b of Table 1, IF--GridNet-She our implemented supervised I'FGGridNet trained on 3-channel AMS-WSJ with 120 epochs (about 1, 1,000,000 steps). In Table 2, TF-GridNNt-Spatiil is our implementation of the supervised TF-GridNet trained on Spatialized WSJO-2Min dataset mentioned in E.2, which tries to generalize to any 3-channel microphone arrays. The model trained for 200 epochs (about 1,000,000 steps) on Spatialized WSJO-2Mix dataset. These models are all trained on single A100 GPU and converges in about 5-6 days.
G. FCP Ablations
The FCP mentioned Sec. 3.2 takes a C1 channel mixture Xe, and a C2 channel source estimate Sk.ca as inputs, and then output the relative RIR Gk 02--01 that can transform Sk.cg to Sk.e
G ,C, Sk.q = Geg-tes Sp.02
(63)
As discussed in Sec. 2, we decided to use the virtual sources as the estimated sources, which more flexible comparing with reference channel source images. Here we list few obvious choices for the virtual source and reason why we use the virtual source based the oracle FCP performance.
First, the virtual source can be the anechoic source signal St- Second, the virtual source can be a partially source signal kk,e in channel c2, where only the first 50 milliseconds of the actual room impulse response is applied to filter the anechoic source 5g. Note that this early reverberant signal motivated by the REVERB challenge (Kinoshita al., 2013), where people found adding the first 50 ms of early reverberation the anechoic signal sounds better. Third and last, since the virtual source is flexible, it can also be the channel c2 's reverberant image Sk.ez-
Now we do few FCP ablations for different kinds of virtual sources. Without loss of generality, we let = 1. Remember the FCP function always takes two signal inputs: the FCP target signal, and the source estimate signal. In Eq. 63, the FCP target is X2, and the source estimate signal is the channel 02 estimate Sk,ey- Here, we do the ablations for different kinds of FCP input combinations, where the source estimate inputs can be the three kinds of virtual sources mentioned above (anechoic, early, reverberant), and the FCP target can either channel c₁ = mixture, or the channel e reverberant source image. Now given the ground- virtual sources and the FCP targets, FCP used estimate the relative RIRs. Then after applying the estimated relative RIR on the virtual source estimate, we get the channel 1 source image estimate $,,1 which is used to calculate the speech metrics in Table 3. We calculate all the results in SMS-WSJ test set.
Table FCP ablations for different kinds of FCP inputs combinations. The metrics reported are channel source images.
Row
Source Estimate
FCP Target
FCP
SDR (dB)
SI-SDR (dB)
PESQ
eSTOI
anechoic Se
mixture (ch-l) X1
FCP(XI, Sy)
22.0
19.8
4.15
0.974
2
early (ch-2) Sk.2
mixture (ch-1) X1
FCP(X,, sqary)
16.5
13.2
3.91
0.943
3
reverb (ch-2) Sk.2
mixture (ch-1) Xi
(t.(X),S).22)
14.7
11.3
3.59
0.917
4
anechoic Še
reverb (ch-1) Sk,1
FCP(S Sx)
34.9
33.3
4.45
0.997
5
early (ch-2) Sk.2
reverb (ch-1) 1
FC Scarly)
18.2
14.5
4.15
0.967
6
reverb (ch-2) Sx.2
reverb (ch-1) Sk,
FCP(St,1,
16.0
12.1
3.84
0.941
First, observe the first three rows in Table 3, where the FCP target is set to be the channel I mixture. In rows 1, 2, and 3, the source estimate is set to be the anechoic source, the early reverberant source, and the reverberant source, respectively. We can see that when the source estimate is set to be the anechoic source, the FCP filtering performance is much better than when the source estimate is early reverberant or reverberant. To see why this is the case, remember that our signal
23

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
model in Eq. 3 in Sec. 2, the relative RIR from channel source image another channel source image involves a inverse filtering operation, which may not exist. This also explains why in row 2a-2b in Table 1, using anechoic speech prior performs better than using the reverberant speech prior.
Row 4,5,6 in Table 3 shows the case when the FCP target is the channel source image. Note that in row 4, when the source estimate is set to groud-truth anechoic source, the metrics is almost perfect, showing the correctness of the signal model used Sec. 2.
H. and Channels Spatialized WSJO-2Mix Evaluation Results
Table Evaluation results for 2-channel Spatialized WSJ0-2Mix. Note that the microphone positions random for this dataset. Top
results emphasized in top1
top2
and top3
Methods denoted with means it is impractical.
Array Agnostic
IVA
Ref.
SDR
SI-SDR (dB)
Row
Methods
Unsup
Prior
Init.
Guide.
(dB)
PESQ
eSTOI
Mixture
0.2
0.0
1.81
0.545
Spatial Cluster
7.9
6.5
2.38
0.689
IVA
Laplace
9.3
8.1
2.52
0.725
lc
IVA
Gaussian
10.9
9.8
2.68
0.770
1d
UNSSOR
V
0.3
-2.7
1.78
0.478
2a
ArrayDPS-A
Anechoic
v
14.5+0.7
13.7=0.7
3.32-+0.88
0.853:40.014
ArrayDPS-D
Anechoic
5.641..0
3.6.
2.34
0.652.L.1114
3a
ArrayDPS-A-Maxx
Anechoic
v
15.3
14.6
3.41
0.870
3b
ArayDPS-D-Max**
Anechoic
X
v
11.2
9.9
2.96
0.799
4a ArrayDPS-A-ML
Anechoic
15.1
14.3
3.39
0.865
4b
ArrayDPS-D-ML.
Anechoic
v
10.4
8.9
2.87
0.773
5a
TF-GrridNet-Spataal
x
(2-mics)
15.2
14.5
3.63
0.888
Table 5. Evaluation results for 3-channel Spatialized WSJ0-2Mix. Note that the microphone positions are random for this dataset. Top
results are emphasized top1
top2
and top3.. Methods denoted with means it is impractical.
Array Agnostic
IVA
Ref.
SDR
SI-SDR (dB)
Row
Methods
Unsup.
Prior
Init.
Guide.
(dB)
PESQ
eSTOI
Mixture
0.2
0.0
1.81
0.545
1a
Spatial Cluster
9.1
7.8
2.53
0.735
lb
IVA
Laplace
10.4
8.5
2.66
0.750
1c
IVA
Gaussian
13.9
12.1
3.07
0.842
ld
UNSSOR
x
1.7
-2.4
1.94
0.519
2a
ArrayDPS-A
Anechoic Anechoic
15.0.10.7 1..7:10.6
3.44
0.872+40.012
2b
ArrayDPS-D
6.3
4.39
2.41
0.66810.111
ArrayDPS-A BoayDDS--A-Mx*
Anechoic
16.4
15.7
3.52
0.886
ArrayDPS-D-Mel
Anechoic
x
12.3
11.0
3.07
0.808
4a
ArrayDPS-A-Mi
Anechoic V <
16.3
15.5
3.49
0.881
4b
ArraDDPS-D-ML
Anechoic
x V
1 11.5
10.0
2.99
0.796
5a
TT-GridNNet-Spatial
(3-mics)
15.5
14.8
3.65
0.892
In addition the 4-channel Spatialized WSI0-2Mix dataset's result in Table 2, we also report 2-channel (first 2 mies) and 3-channel (first 3 mics) results on Spatialized WSJO-2Mix dataset, in Table and Table 5 respectively.
24

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
For the 2-channel case, we first note that in row 1d of Table 4, the UNSSOR does work properly. This is actually as expected because UNSSOR assumes over-determined scenario (Wang & Watanabe, 2023), which means the number of microphones should be larger than the number of speakers. In contrast, our ArrayDPS is able to work properly, In addition, our method easily outperforms any other unsupervised methods. Finally, we compare row 5a with row 2a and row 4a, we find that the supervised method is slightly better than ArrayDPS-AMMI. in all metrics for the 2-channel case.
For the 3-channel results shown Table 5, the result is very similar to the 4-channel results shown in Table In short, ArrayDPS-A outperforms all other unsupervised methods, while UNSSOR does not work properly in this 3-channel setting, As mentioned Sec. F, UNSSOR is very sensitive ISMS loss weight y, but we still cannot find a working weight after an extensive parameter search. Note that ensure that our training is correct it works for both 4-channel Spatialized WSJO-2Mix and 3-channel SMS-WSJ datasets. One explanation is that the training dataset contains samples recorded by different 3-channel ad-hoc microphone arrays, which makes it more difficult for UNSSOR to converge. We further compare ArrayDPS (row 2a, 4a) with the supervised baseline (row Sa), where ArrayDPS-A mean SDR and SI-SDR can outperform the supervised JF-GridNet-Spatiaal
6-Channel JSS-WSJ Evaluation Results
In addition to the 3-channel SMS-WSJ result reported in Table 1, also report full 6-channel (all mics used) result for SMS-WE SSSJ dataset. We first compare row 1(a-c) with row 2a, where our ArrayDPS-A easily outperforms spatial clustering and IVA. Comparing row 1d with row 2d, we can see that ArrayDPS-A'Ss mean metric score outperforms UNSSOR by about 0.7 dB in SDR and SI-SDR, but performs a bit worse in terms of PESQ and eSTOL. However, our maximum likelihood version ArrayDPS- AMML (row 4a) casily outperforms UNSSOR in metrics by a large margin. Although 6-channel setting, APrayDSS- ---OPPS--MMMM A-ML still better than any other unsupervised methods, the supervised T'F-Gridee-SSM. in row 5a leads our APS by a large margin. Future research will be conducted reduce this performance gap.
Table Evaluation results for 6-channel SMS-WSJ. Methods denoted with f are results from UNSSOR (Wang Watanabe, 2023), and methods denoted with mean it is impractical. Note that SMS-WSS BSS-WSJ only contains samples with a tixed microphone array Top results are emphasized in top1 top2 and top3...
Array
IVA
Ref.
SDR (dB)
SI-SDR (dB)
Row
Methods
Unsup
Agnostic
Prior
Init.
Guide.
PESQ
eSTOI
Mixture
0.1
0.0
1.87
0.603
la
Spatial Cluster
11.9
10.2
2.61
0.735
lb
IVAt
V
Laplace
10.6
8.9
2.58
0.764
lc
IVA
/
V
Gaussian
14.7
13.4
3.07
0.865
ld
UNSSOR
x
15.7
14.7
3.47
0.884
2a
ArrayDPS-A
Anechoic
16.3 1.2
15.4+ 1.3
3.45++0.12
0.873++0.019
2b
ArrayDPS-D
Anechoic
/
8.9
7.2+5.44
0.67
0.743++0.115
3a
ArrayDP-A-MMa
v
Anechoic
v
17.6
16.9
3.59
0.896
ArraYDPS-D-Max*
Anechoic x
15.1
14.0
3.32
0.873
4a
ArrayDPS-A-Ma
V
Anechoic V
<
17.4
16.6
3.56
0.891
4b
ArrayDPS-D-ML
V
Anechoic x
<
14.6
13.5
3.27
0.862
5a
TT-SridNet-SMSt
19.4
18.9
4.08
0.949
J. 3-Speaker Evaluation Results
For 3-speaker source separation, we evaluate on the spatialized WSJ0-3Mix dataset, which is the 3-speaker version of the ad-hoc spatialized WSJ0-2Mix dataset. We evaluate on the first 4 microphones, where the results are shown in Table 7. As shown in Row 2a-5a, ArrayDPS outperforms the supervised baseline by a large margin in all metrics. If we sample five samples and then take the maximum likelihood one (Row 4a), ArrayDPS outperforms supervised TF-GridNNe-SSpatin by more than 3 dB in terms of SDR and SI-SDR. This shows the potential of ArrayDPS in more diverse acoustic scenarios.
25

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Table Evaluation results for 3-speaker reverberant speech separation 4-channel Spatialized WSJO-3Mix. Note that the microphone positions random for this dataset. Top results are emphasized in topl impractical.
top2
and top3 Methods denoted with means it
Array
IVA
Ref.
SDR (dB)
SI-SDR (dB)
Row
Methods
Unsup
Prior
PESQ
eTTII
Agnostic
Init.
Guide.
Mixture
-3.0
-3.3
1.50
0.371
Spatial Cluster
6.9
5.6
2.10
0.625
IVA
Laplace
5.9
3.8
2.10
0.574
IVA
Gaussian
9.6
7.6
2.46
0.701
1d
UNSSOR
-2.8
-6.1
1.50
0.313
2a
ArrayDPS-A
Anechoic
v
12.8 +: 1.2
11.8 1.3
3.1110.16
0.791110.028
ArrayDPS-A-Man
Anechoic
14.3
13.4
3.31
0.816
4a
ArrayDPS-A-Mi
Anechoic
14.0
13.0
3.27
0.807
TT-aaia
(4-mics)
10.6
9.7
2.85
0.747
K. Ablations on Sensitivity to Hyperparameters
To evaluate our ArrayDPS's sensitivity and robustness to hyper-parameters, we conduct ablation experiments on the likelihood score guidance ç, the sampling starting noise level Tmax, FCP filter length F, the number steps N to use the IVA initialized filters, whether to use Gaussian or Laplace prior for IVA initialization, and whether use the IVA separated source diffusion initialization (line 8 of Algorithm 1). For each parameter, do ablations by modifying it starting from the default parameter mentioned in Appendix F. All the ablations conducted on the first 50 samples of the SMS-WSJ validation set. Results are shown in Table 8-13.
Table 8. Ablation study for &.
5
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
SI-SDR
15.2
11.5
15.6
1..6
15.3
15.0
15.0
15.2.12.6
15.1
Table 9. Ablation study for Tmax-
Tmax
0.5
0.6
0.7
0.8
0.9
1.0
1.1
SI-SDR
16. +6.9
16.0
1..
15.6
15.1
14.8
14.3
Table 10. Ablation study for FCP filter longth F.
F
10
11
12
13
14
15
16
SI-SDR
1.38
1.65
1.2
15.1
14.9+1.7
14.9+1.5
14.8:11.6
Table 11. Ablation study for filter guidance steps Nşg-
Njg
70
80
90
100
110
120
130
SI-SDR
15.0++1.7
15.2+ 1.7
14.6+-2.1
15.2+11.7
15.4+112
15.5+1111
15.5++1.1
26

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Table 12. Ahlation study for IVA prior used for IVA initialization.
IVA Prior
Gaussian
Laplace
SI-SDR
15.3+1.6
14.8+1.6
Table 13. Ablation study for using IVA separated sources diffusion initialization (line of Algorithm 1).
Diffusion Initialization
Yes
No
SI-SDR
1..3+1.6
13.3+2.3
L. Filter and Source Visualization
Figure 3- 6 gives visualization of the separated sources and the final FCP estimated relative RIRs. In each figure of Figure 3-6 6, the first row contains the ground-truth anechoic source $1. the ground-truth RIR h1,1 (from anechoic source to reference channel), and the reference-channel reverberant source 81,1- Obviously, the third signal is the convolution of the first signals. Then the second shows the same for our ArrayDPS, i.e., separated virtual source 8o. the final FCP estimated relative RIR - and the separated reference-channel source $1,1- These two adjacent rows allow direct comparison between the ground-truth and ArrayDPS. Similarly, row 3 and row show results for source 2.
These figures show us the difference between the final FCP estimated filter and the ground truth RIR. Those two signals might not be aligned, but sometimes show similar structure on the spectrogram. On the other hand, the virtual source separated by ArrayDPS more like the anechoic signal rather than the reverberant signal. We explain this for two reasons: the diffusion model is trained mostly on clean anechoic speech, hence the inclination to generate anechoic sources; 2) FCP performs much better when the input source signal is anechoic (see Appendíx G). This means outputting anechoie source (instead of reverberant onc) would satisfy a higher likelihood.
Our demo site also shows virtual sources with 2-speaker separation. We urge readers to listen to the virtual source samples and compare them with the final output and the ground-truth anechoic source. It appears that ArrayDPS is indeed accomplishing some dereverberation, however, we do not want to over-claim here and intend to verify this promise in future research.
27

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Groundtruth Anechoic
Groundtruth fn.
Groundtruth Reverberant 51,1
ArrayDPS Separated $ (virtual)
FCP Estimated (
ArrayDPS Separated 5,, (reverb)
Groundtruth Anechoic 5,
Groundtruth RIR h
Groundtruth Reverberant $2.1
ArrayDPS Separated (x(irtual))
FCP Estimated gh...
ArrayDPS Separated 52 (reverb)
Figure 3. SMS-WSJ Sample 14420040_443304QQ0 Visualization
Groundtruth Anechoic 5
Groundtruth RIR h
Groundtruth Reverberant $1.1
ArrayDPS Separated (virtual)
Estimated g
ArrayDPS Separated 5 (reverb)
Groundtruth Anechoic
Groundtruth RIR h
Groundtruth Reverberant
ArrayDPS Separated (y(irtuall)
FCP Estimated g
ArrayDPS Separated 5, (reverb)
Figure 4. SMS-WSJ Sample 1015 446c0415 442c040c Visualization
28

---

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Groundtruth Anechoic
Groundtruth fg.
Groundtruth Reverberant 51,1
ArrayDPS Separated 5;
FCP Estimated .
ArrayDPS Separated 5
Groundtruth Anechoic 5,
Groundtruth RIR h
Groundtruth Reverberant $2.1
ArrayDPS Separated š
FCP Estimated go...
ArrayDPS Separated 5z
Figure 5. NSS-WSJ Sample 1120,4450000.444c00000 Visualization
Groundtruth Anechoic
Groundtruth RIR h
Groundtruth Reverberant $1.1
ArrayDPS Separated (virtual)
Estimated g
ArrayDPS Separated 5 (reverb)
Groundtruth Anechoic 5
Groundtruth RIR h
Groundtruth Reverberant $2
ArrayDPS Separated (x(irtuall)
FCP Estimated g
ArrayDPS Separated 5, (reverb)
Figure 6. SMS-WSJ Sample 999 441c040c 447c040k Visualization
29


Reviews:
[{"review_id": "d7JuEce7Wc", "reviewer": "Reviewer_Go2n", "summary": "The paper proposes a method for blind source separation in scenarios where the microphone array and RIR are unknown. The authors adopt a virtual source model, mapping all sources to an imaginary reference microphone.\nThey formulate the separation problem as an inverse problem, where: (i) a single-source diffusion model serves as the prior; (ii) The unknown likelihood, which involves the RIR, is estimated in the frequency domain.\nThe method is unsupervised and relies on diffusion posterior sampling.\nFinally, the authors compare their approach against both supervised and unsupervised methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and constructive comments.\n> Q: The advantage of using a virtual source model is unclear and questionable.\n\nYes, we understand this concern now. We had tried to explain this but the explanations are scattered in the paper (lines 147-152 (column 1), line 259 (column 2), and then again in Appendix G). We will consolidate these explanations in one place. Briefly, our main reason to propose a virtual source model is as follows:\n\n-- When FCP computes the relative RIR even with respect to the ground truth (reverberant) signal measured at one channel, FCP fails to reconstruct any other channels' sources perfectly (only 11.3 dB of SI-SDR as shown in Table 3).\n\n-- The reason is the relative RIR filter, as mentioned in Eq. 3 (lines 98-99), involves an inverse filtering, for which the inverse may not exist, and may also make the relative filter non-causal. We discuss this in Appendix G (lines 1143-1144).\n\n-- On the other hand, Table 3 row 1 shows that FCP computes the relative RIRs much better when the source estimate is the original anechoic source.\n\nHence, we proposed to use the virtual source model, hoping that we can sample the anechoic source, thereby leveraging the effectiveness of FCP. We apologize for the confusion; we will explain this in detail early in the paper.\n\n> Q: The term intractable is misused to describe the likelihood.\n\nWe think we understand the issue. In previous literature like DPS[1], $p(y|x_t)$ is usually viewed as intractable because it needs to integrate through $x_0$, or $p(x_0|x_t)$ is intractable because it needs to integrate through all intermediate variables. In our case, we viewed $p(y|x_0)$ as intractable because it needs to integrate through all possible operators on $x_0$. Perhaps this was the source of the confusion. We still think that the intractable term applies to our case, but it needs to be appropriately explained. We will modify the paper accordingly.\n\n>Q: The experiments were conducted with only two speakers. More complex scenarios should be tested.\n\nWe ran new experiments with 3 speakers on the spatialized WSJ0-3Mix dataset (reverberant) [2]. Below is our result table on the 3-speaker evaluation.\nWe also recorded real-world mixtures for the demo.\n**The demo page https://arraydps.github.io/ArrayDPSDemo/ now contains 3-speaker demo and real-world mixtures demo.** More real-world demo samples can be found in https://github.com/ArrayDPS/ArrayDPSDemo/tree/main/demo_results_realworld.\n| Method               |   SDR     |  SI-SDR   |   PESQ     |   ESTOI     |\n|----------------------|-----------|-----------|------------|-------------|\n| Mixture              |  -3.0     |  -3.3     |  1.50      |  0.371      |\n| IVA (Laplacian)      |   5.9     |   3.8     |  2.10      |  0.574      |\n| IVA (Gaussian)       |   9.6     |   7.6     |  2.46      |  0.701      |\n| SpatialCluster       |   6.9     |   5.6     |  2.10      |  0.625      |\n| UNSSOR               |  -2.8     |  -6.1     |  1.50      |  0.313      |\n| ArrayDPS-A           |  12.8 ± 1.2 | 11.8 ± 1.3 | 3.11 ± 0.16 | 0.791 ± 0.028 |\n| ArrayDPS-A-ML        | **14.0**  | **13.0**  | **3.27**   | **0.807**   |\n| ArrayDPS-A-Max       |  14.3     |  13.4     |  3.31      |  0.816      |\n| TF-GridNet-Spatial   |  10.6     |   9.7     |  2.85      |  0.747      |\n\nFor the 3-speaker WSJ0-3Mix dataset [2], we are pleasantly surprised that ArrayDPS-A outperforms supervised TF-GridNet-Spatial by a large margin. This shows ArrayDPS's potential in more complicated scenarios.\n\n>Q: Missing citation of BlindDPS[3].\n\nThis definitely should be cited. We were supposed to cite this in line 421, second column, when we discussed the blind inverse problem with DPS, but we mistakenly cited DPS again there. Sincere apologies, and thank you for pointing this out.\n\n>Q: Typos in line 211 (confusion for S), second column, line 158-164, second column (missing log), Eq.6 (minus sign), line 136-137, second column (Gaussian typo), Eq.13  typo, line 20 of algorithm 1 (wrong args in get_score).\n\nThanks! We will fix all these typos in the next version.\n\n>Q: Algorithm 1 is too complex and has 2 return statements.\n\nThanks for this suggestion. We will abbreviate algorithm 1 and abstract out the second-order Heun sampler. We will also return both signals in the end.\n\n[1] Chung, Hyungjin, et al. *Diffusion posterior sampling for general noisy inverse problems.* arXiv preprint arXiv:2209.14687 (2022).\n\n[2] Mitsubishi Electric Research Laboratories. *Deep Clustering*. https://www.merl.com/research/highlights/deep-clustering\n\n[3] Chung, Hyungjin, et al. *Parallel Diffusion Models of Operator and Image for Blind Inverse Problems*. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 1, No. 2, 2023.", "claims_and_evidence": "- The authors present the virtual source model as a flexible framework, but its utility remains unclear throughout the paper. When a source is among the channels, the framework reduces to the relative model, making its advantages questionable.\n- The term intractable is misused to describe the likelihood. The issue is not computational hardness but rather the inability to evaluate the likelihood due to the unspecified array geometry. This misuse is further misleading, as intermediate likelihoods are generally intractable; see Equation (7) in [1] and the subsequent explanation.\n\n---\n\n.. [1] Chung, Hyungjin, et al. \"Diffusion posterior sampling for general noisy inverse problems.\" arXiv preprint arXiv:2209.14687 (2022).", "methods_and_evaluation_criteria": "- The evaluation methods are fair, with one exception: the experiments were conducted with only two speakers.\nThe method should also be tested in a multi-speaker setup to assess its performance in more complex scenarios.\n- Additionally, the computational requirements were not evaluated.\nGiven that the procedure involves likelihood estimation, it is important to assess the computational cost of blind speech separation and contrast it with supervised methods.", "theoretical_claims": "The theoretical claims in the main paper are correct.", "experimental_designs_or_analyses": "The experimental design is sound, except for the limitation to only two speakers.\nThe method should be tested in a multi-speaker setup to better evaluate its effectiveness in more complex separation tasks.", "relation_to_broader_scientific_literature": "In Table 2, where the microphone array is unknown, the proposed method shows only marginal improvement over supervised methods.\nThis contradicts the authors' claim that supervised methods lack flexibility and fail to generalize to unseen geometries.\nIn fact, the results suggest the opposite: the supervised algorithm TF-GridNet-Spatial performs nearly as well as the proposed approach.", "essential_references_not_discussed": "The seminal work of [1] where that introduces blind diffusion posterior sampling was not mentioned.\n\n---\n.. [1] Chung, Hyungjin, et al. \"Parallel diffusion models of operator and image for blind inverse problems. IEEE.\" CVF Conference on Computer Vision and Pattern Recognition. Vol. 1. No. 2. 2023.", "other_strengths_and_weaknesses": "- There is confusion in the notation, particularly for the denoiser (Col-2, Line 211), where \"s\" is uppercase.\nThis is misleading since uppercase letters seem to be reserved for frequency-domain quantities, yet here it is used to denote the score.\n- Algorithm 1 is overly complex: Lines 10 to 25 essentially implement the Heun sampler from EDM, see Algorithm 2 in [1], with the only difference being how the conditional scores (and thus the denoiser) are computed.\nIt would be clearer to simply state that the algorithm applies the EDM solver with the denoiser described in Algorithm 2, rather than explicitly rewriting these steps.\n\n\n---\n\n.. [1] Karras, Tero, et al. \"Elucidating the design space of diffusion-based generative models.\" Advances in neural information processing systems 35 (2022): 26565-26577.", "other_comments_or_suggestions": "Algorithm 1 Issues:\n- Contains two ``return`` statements, which may cause confusion.\n- the function ``GET_SCORE`` in Line 20 has incorrect arguments. Specifically, it does not take the denoiser as input according to its definition in Algorithm 2\n\nOther Errors:\n- Col-2, Lines 158-164: the $\\log$ were forgotten before $p$.\n- Equation (6): Incorrect SDE; missing a negative sign.\n- Gaussian Definition: Incorrect; should have mean 0 and covariance $\\sigma^2 I$\n- Equation (13): Should use spatial convolution $*_l$", "questions_for_authors": "I have no further question for the authors.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V37u4WYPCx", "reviewer": "Reviewer_KMQX", "summary": "The paper introduces ArrayDPS, an unsupervised, generative, and array-agnostic method for blind speech separation (BSS). ArrayDPS leverages a diffusion-based generative model as a speech prior, combined with a novel approximation method for the intractable likelihood arising due to unknown array geometries and unknown room impulse responses (RIR). The key conceptual innovation is employing Diffusion Posterior Sampling (DPS) combined with the Forward Convolutive Prediction (FCP) algorithm, allowing estimation of relative room impulse responses (RIRs). Notably, ArrayDPS achieves competitive performance with supervised approaches while significantly surpassing previous unsupervised techniques. Experiments are performed on standard benchmarks, SMS-WSJ (fixed array) and Spatialized WSJ0-2Mix (ad-hoc array scenarios), showcasing ArrayDPS’s generalization across different microphone arrays and acoustics. The paper demonstrates a notable advancement in unsupervised multi-channel speech separation by integrating strong priors learned from single-channel diffusion models with likelihood approximations tailored for blind array conditions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and constructive comments.\n>Q: The paper could strengthen its evidence by evaluating more diverse real-world scenarios or noisy acoustic conditions to demonstrate true generalizability. Have you considered evaluating ArrayDPS on realistic datasets involving more complex acoustic conditions (real recordings, background noise, and dynamic speakers)? Could this impact your algorithm's stability and robustness?\n\nThank you for the suggestion. We ran new experiments for the following scenarios:\n(1) Real-world mixture recordings in a 7m x 4m x 2.7m room, where 2 speakers speak simultaneously. The recordings are performed using 3 microphones and under weak background noise.\n(2) A 3-speaker dataset to test the separation performance of all 3 speakers.\nThe real-world experiment maintains strong separation performance, while in the 3-speaker (synthetic dataset) experiment, ArrayDPS outperforms the baselines by a large margin (to our surprise). The main results are reported here: **https://arraydps.github.io/ArrayDPSDemo/**, and additional demos and recording files are available here: **https://github.com/ArrayDPS/ArrayDPSDemo/tree/main/demo_results_realworld**.\n\n>Q: Sensitivity to hyperparameters like diffusion schedule, IVA parameters, and RIR estimation settings?\n\nThanks for raising this question. We ran some new ablations for six important parameters, namely:\n(1) the diffusion scheduler’s starting noise level $\\tau_{\\max}$,\n(2) IVA filter guidance steps $N_{fg}$ in Algo.2 line 9,\n(3) likelihood guidance $\\xi$ in Eq. 57,\n(4) causal FCP filter length $F$,\n(5) initialized with IVA-Gaussian or IVA-Laplace (row 1b and 1c in Table 1), and\n(6) whether the warm initialization in line 8 of Algorithm 1 is used.\nSince inference is expensive, we show SI-SDR only for the first 30 mixtures of the 3-channel SMS-WSJ’s validation set (we sample each mixture 5 times). The default parameters are $\\tau_{\\max}=0.8, N_{fg}=100, F=13, \\xi=2$ and IVA-Gaussian and warm initialization are used.\nFor results below, we change one parameter at a time.\n\n|$\\xi$|1.2|1.4|1.6|1.8| 2.0|2.2|2.4|2.6|2.8|\n|-|-|-|-|-|-|-|-|-|-|\n|SI-SDR|15.2±1.1|15.5±1.1|15.6±1.0|15.6±1.2|15.3±1.6|15.0±1.5|15.0±1.6|15.2±2.6|15.1±1.8|\n\n|$\\tau_{max}$|0.5|0.6|0.7|0.8|0.9|1.0|1.1|\n|-|-|-|-|-|-|-|-|\n|SI-SDR|16.1±0.9|16.1±0.8|15.6±1.3|15.3±1.6|15.1±1.7|14.8±1.8|14.3±2.1|\n\n|$F$|10|11|12|13|14|15|16|\n|-|-|-|-|-|-|-|-|\n|SI-SDR|15.8±1.3|15.5±1.6|15.6±1.5|15.3±1.6|14.9±1.7|14.9±1.5|14.8±1.6|\n\n|$N_{fg}$|70|80|90|100|110|120|130|\n|-|-|-|-|-|-|-|-|\n|SI-SDR|15.0±1.7|15.2±1.7|14.6±2.1|15.3±1.6|15.2±1.7|15.4±1.2|15.5±1.1| \n\n|Initialization used|IVA-Gaussian|IVA-Laplace|\n|-|-|-|\n|SI-SDR|15.3±1.6|14.8±1.6|\n\n|Use Warm Initialization|Yes|No|\n|-|-|-|\n|SI-SDR|15.3±1.6|13.3+2.3|\n\nWe believe the proposed algorithm is robust to most parameters, although the result is only on a 30-mixture subset. To make this more conclusive, we will finish the ablation on a larger dataset.\n\n>Q: Computational complexity for real-time deployment scenarios and possible strategies for optimization.\n\nCurrently, ArrayDPS and all neural baselines in this paper are impractical for real-time deployment scenarios due to the non-causal processing and high computation requirements. However, we think there are opportunities to mitigate this hurdle. First, we can use diffusion distillation techniques to reduce sampling steps or even make it one step. Also, our algorithm can easily fit into the flow matching’s sampling ODE, which would be more efficient. To enable real-time processing, we need to modify the diffusion U-Net to be a causal architecture, and then update FCP and the likelihood calculation with an adaptive version; we expect this to be reasonably straightforward.\n\n>Q: The algorithm heavily relies on IVA initialization for stability and performance. This dependency somewhat reduces the claimed elegance and universality of the method.\n\nWe agree with this. Currently, ArrayDPS is sensitive to initialization, bearing similarity to the Expectation Maximization (EM) algorithm. We will add a section to discuss this and also give insights for future directions (1. Inducing a filter prior during sampling. 2. In early steps, maintain some uncertainty for the filter instead of greedily optimizing using FCP).\n\n>Q: The result STD (around ±1.2 dB) exhibits notable instability, potentially limiting reliability.\n\nWe respectfully disagree with this argument since our method is generative, which means that our method samples multiple plausible solutions as separation results. These solutions might differ at the sample level, but all sound plausible. We believe (±1.2dB) is a reasonable range for 5 samples (please verify this in our demo about how all 5 different samples sound correct, but have ±1.2dB fluctuation). However, without IVA initialization, ArrayDPS-D exhibits a fluctuation of ±4.6 dB, implying our algorithm lacks consistency without IVA.\n\nThanks again for your insightful observations.", "claims_and_evidence": "The primary claim is that ArrayDPS achieves competitive performance compared to supervised methods and outperforms unsupervised methods significantly. This claim is reasonably supported by comprehensive experiments on recognized benchmark datasets (SMS-WSJ, Spatialized WSJ0-2Mix). However, the paper could strengthen its evidence by evaluating more diverse real-world scenarios or noisy acoustic conditions to demonstrate true generalizability. Additionally, the reproducibility of the likelihood approximation and posterior sampling processes should be validated more thoroughly with sensitivity analysis or additional controlled experiments.", "methods_and_evaluation_criteria": "**Proposed Methods**\n- The paper applies an existing DPS framework but modifies the likelihood term to handle multi-channel signals. The authors approximate the “unknown” part of the likelihood with a linear filter estimation approach (Forward Convolutive Prediction), thereby modeling room acoustics implicitly.\n- They combine a generic scoring from a single-speaker diffusion model with a relative RIR fitting routine in each sampling step. This step yields a multi-channel separation pipeline that does not require supervised training on a matched array geometry.\n\n**Evaluation Criteria**\n- The paper focuses on standard separation metrics: SDR, SI-SDR, PESQ, and eSTOI. They compare the mean (and also best sample) across multiple runs, which is helpful in understanding both typical performance and best-case scenario.\n- They use well-established corpora (SMS-WSJ, Spatialized WSJ0-2Mix) that test both fixed and ad-hoc array setups, thus covering a broad range of real-world scenarios.\n\nOverall, the approach and benchmarks are appropriate. The reliance on standard speech separation metrics is conventional, though it might have been beneficial to analyze more robust real-room data or consider time/frequency domain direct speech intelligibility measures. The methodology is consistent with standard practices in speech separation research.", "theoretical_claims": "The paper primarily provides empirical contributions without heavy theoretical claims. However, the authors provide a theoretical justification for the equivalence of FCP to maximum likelihood estimation in the supplementary derivation (Theorem B.1). The derivations in Appendix B are clearly stated and appear mathematically correct and consistent.", "experimental_designs_or_analyses": "The experimental design is strong and robust. It clearly explained experiments and configurations are detailed thoroughly, and extensive ablation studies (e.g., the effect of IVA initialization, anechoic vs reverberant priors, and reference-channel guidance) are performed. Also, the empirical approximation assumptions are explicitly analyzed and justified.", "relation_to_broader_scientific_literature": "ArrayDPS connects clearly with broader literature by **Extending Diffusion Posterior Sampling (DPS)**, previously employed in single-channel or supervised multi-channel scenarios, to unsupervised and array-agnostic BSS. It also provides a bridge from classic spatial clustering, ICA, IVA methods, and recent neural generative approaches (e.g., diffusion-based generative models).", "essential_references_not_discussed": "The paper provides an extensive reference list.", "other_strengths_and_weaknesses": "**Strengths**\n\n- ArrayDPS demonstrates notable originality in applying generative diffusion priors to a challenging unsupervised multi-channel blind separation scenario.\n- The proposed framework elegantly avoids needing explicit microphone geometry knowledge, significantly improving usability and practical applicability to diverse hardware configurations.\n\n**Weaknesses**\n\n- The algorithm heavily relies on IVA initialization for stability and performance. This dependency somewhat reduces the claimed elegance and universality of the method.\n- The standard deviation of the results (around ±1.2 dB) indicates that the method still exhibits notable instability, potentially limiting practical reliability.", "other_comments_or_suggestions": "Please refer to the previous sections.", "questions_for_authors": "- Have you considered evaluating ArrayDPS on realistic datasets involving more complex acoustic conditions (real recordings, background noise, and dynamic speakers)? Could this impact your algorithm's stability and robustness?\n\n- How sensitive is ArrayDPS to variations in hyperparameters such as diffusion schedule, IVA parameters, and RIR estimation settings?\n\n- Is the computational complexity of ArrayDPS practical for real-time deployment scenarios? If not, could you briefly describe possible strategies for optimization or simplification?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0SZDNMVrAn", "reviewer": "Reviewer_L5PQ", "summary": "This paper proposes ArrayDPS for multi-channel blind speech separation. The main contributions claimed by the authors are as follows:\n1. Proposal of an array-agnostic unsupervised learning-based blind speech separation model that is robust to microphone array variations.\n2. Introduction of the first method utilizing Diffusion Posterior Sampling (DPS) to solve the multi-channel array inverse problem.\n3. Achieving performance comparable to supervised methods while surpassing existing unsupervised approaches in the field of speech separation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and constructive comments. New results from real-world experiments and a 3-speaker dataset are discussed in the rebuttal to reviewer Go2n. Corresponding demos are updated in **https://arraydps.github.io/ArrayDPSDemo/**.\n\n>Q: GET_SCORE(.) in Algorithm 2 contains wrong arguments.\n\nThis is a typo, where $D_\\theta$ should be removed and $\\hat{G}^{k, \\text{IVA}}_{1\\rightarrow c}$ should be inside the parentheses. We are sincerely sorry and will fix it.\n\n>Q: The dataset relies on simulated RIRs, raising generalization concerns.\n\nTo validate our method in real-world conditions, we recorded 15 mixtures in a 7m x 4m x 2.7m room. Two volunteers speak simultaneously with weak environmental noise and are recorded by 3 microphones. Demos and recordings are in https://github.com/ArrayDPS/ArrayDPSDemo/tree/main/demo_results_realworld and are updated in the demo site.\n\n>Q: The \"Max\" model is impractical and not fair for comparison.\n\nThanks for pointing this out. We will remove it from the ranking highlight and mark those as impractical.\n\n>Q: The array-agnostic baseline model used is relatively outdated, and TF-GridNet-Spatial is merely a supervised model trained on diverse microphone arrays. Comparison with a more relevant array-geometry-robust model, such as VarArray [1], should be considered.\n\nWe understand the concern that TF-GridNet [2], although a very strong baseline, is not designed in an array-agnostic manner. However, we do not think this TASLP 2023 SOTA paper is outdated due to its inability to handle any number of microphone inputs. Training on 3-channel ad-hoc arrays could result in better performance when evaluated on the same 3-channel scenarios. Of course, this needs further research for validation as there is no existing array-agnostic version of TF-GridNet for direct comparison.\n\nFor the VarArray baseline, we are a bit unsure about it as: (1) it follows FaSNet-TAC [3], but only reports word error rate and does not compare with FaSNet-TAC, (2) recent work [4] shows that it performs worse than FaSNet-TAC, and (3) to the best of our knowledge, there is no open-source implementation.\n\nIn light of this, we directly trained an array-agnostic FaSNet-TAC on WSJ0-2Mix with a variable number of channels, and report the separation results below:\n|FaSNet-TAC|mics|SDR|SI-SDR|PESQ|ESTOI|\n|-|-|-|-|-|-|\n|Spatialized-WSJ|2|8.2|7.4|2.26|0.663|\n|Spatialized-WSJ|3|9.5|8.6|2.37|0.689|\n|Spatialized-WSJ|4|9.9|9.0|2.41|0.700|\n|SMS-WSJ|3|9.3|8.5|2.29|0.704|\n|SMS-WSJ|6|9.5|8.7|2.33|0.711|\n\nWe understand that this is by no means a fair baseline as it's using a 2020 backbone network. We will further look into a TAC version of TF-GridNet to fully clarify this issue.\n\n>Q: The impact of IVA initialization on performance is significant but lacks analysis.\n\nReviewer KMQX also mentioned this, so we refer to our rebuttal to KMQX, where we report three ablations around the IVA initialization. We first compare  ArrayDPS initialized by IVA-Gaussian and IVA-Laplace. The second ablation is for parameter $N_{fg}$ (Algo. 2 line 9), which is the initial number of steps to use the IVA initialized filter. The third ablation compares using warm initialization or not in Algorithm 1, line 8. We believe these are the most important parameters to analyze, as the first shows ArrayDPS's sensitivity to IVA performance (Laplace prior performs worse), the second shows at what sampling time we should stop using the initialization and let FCP learn the filter, and the last one shows the effectiveness of initializing the diffusion starting point with IVA.\n\n>Q: Reference formatting issues\n\nThanks! We realized there are inconsistencies and issues in the references. We will fix them.\n\n>Q: Related work should be placed earlier.\n\nThanks for the suggestion! We thought about this but made the call to move it to the end primarily because it would contain background material from multiple domains, including unsupervised separation, diffusion posterior sampling, and RIR estimation, etc. We thought a long related work may interrupt the flow of the paper.\n\n>Q: line 739 has an extra \")\", \"row1,2,3\" → \"rows 1, 2, and 3\", \"Table G\" → \"Table 3\", \"Algorithm 3.5\" → \"Sec. 3.5\"\n\nApologies. We will fix all these typos.\n\n[1] Yoshioka, Takuya, et al. \"VarArray: Array-geometry-agnostic continuous speech separation.\" ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022.\n\n[2] Z.-Q. Wang, et al. \"TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation,\" IEEE/ACM TASLP, vol. 31, pp. 3221–3236, 2023.\n\n[3] Luo, Yi, et al. \"End-to-end microphone permutation and number invariant multi-channel speech separation.\" ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2020.\n\n[4] Chen, Weiguang, et al. UniArray: Unified Spectral-Spatial Modeling for Array-Geometry-Agnostic Speech Separation. arXiv preprint arXiv:2503.05110, 2025.", "claims_and_evidence": "Strengths\n1. The limitations of conventional supervised learning-based approaches using synthetic dataset are well-identified, and the formulation of blind speech separation as an inverse problem is clearly structured.\n2. The overall overview of the proposed ArrayDPS using DPS is well summarized, and both the detailed explanation and derivation of the posterior score approximation using the prior score and likelihood score are clearly presented.\n3. As an unsupervised and array-agnostic approach in the field of speech separation, ArrayDPS surpasses existing limitations and demonstrates its potential applicability to real-world applications.\n\nWeaknesses\n1. Limitations in the experiments conducted to support the proposed claims. (detailed in the “Experimental Designs or Analyses”)\n2. Minor issue related to inconsistencies in mathematical formulation and function usage. (detailed in the “Theoretical Claims”)\n3. Minor issues related to the paper structure and typos. (detailed in “Other Comments or Suggestions”)", "methods_and_evaluation_criteria": "The evaluation metrics used for blind speech separation—SDR, SI-SDR, PESQ, and eSTOI—seem appropriate and reasonable for assessing model performance.", "theoretical_claims": "The process of estimating the likelihood score from the log likelihood approximation and the prior score from the diffusion denoising model to compute the posterior score in the ODE is logically well-explained. Furthermore, the Appendix clearly presents the derivation, analysis and validation of the posterior score approximation.\n\nHowever, in Algorithm 1, the function GET_SCORE(.) calls Algorithm 2, but the input used in Algorithm 1 (line 20) differs from the input specified in Algorithm 2, making its usage unclear.", "experimental_designs_or_analyses": "Strengths\n1. The distinction between \"unsupervised\" and \"array-agnostic\" models in the Tables enhances the clarity and intuitiveness of the results.\n2. The robustness of the proposed method to microphone array variations is well-supported through experiments conducted on random microphone arrays (Spatialized WSJ0-2mix dataset).\n\nWeaknesses\n1. The authors point out the generalizability issue from supervised learning-based approaches using synthetic dataset. However, the dataset used in this paper also relies on simulated RIRs, raising the same generalization concerns.\n2. The \"Max\" model selects the best of five samples for each test set, which may not be a fair comparison. It would be helpful to explicitly state in the Table whether this approach is “practical”, and consider refraining from using it for ranking in the top results to ensure a fair evaluation.\n3. The array-agnostic baseline model used is relatively outdated, and TF-GridNet-Spatial is merely a supervised model trained on diverse microphone arrays. Comparison with a more relevant array-geometry-robust model, such as VarArray [1], should be considered.\n4. The impact of IVA initialization on performance is significant, yet there is a lack of detailed analysis regarding its contribution.\n\n[1] Yoshioka, Takuya, et al. \"VarArray: Array-geometry-agnostic continuous speech separation.\" ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022.", "relation_to_broader_scientific_literature": "1. Unlike conventional methods that rely on fixed microphone arrays, the proposed array-agnostic speech separation method offers greater flexibility for real-world applications.\n2. The performance improvement in unsupervised approaches suggests potential applications in other unsupervised learning domains.", "essential_references_not_discussed": "As mentioned in “Experimental Designs Or Analyses”, it is necessary to consider existing array-agnostic specialized models, such as VarArray [1].\n\n[1] Yoshioka, Takuya, et al. \"VarArray: Array-geometry-agnostic continuous speech separation.\" ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022.", "other_strengths_and_weaknesses": "As mentioned above, the problem formulation and the proposed solution are clearly presented. However, the experimental validation and analysis are insufficient to fully support the proposed claims.", "other_comments_or_suggestions": "1. Reference formatting issues:\n - The citation format for ICASSP conference papers is inconsistent.\n - Capitalization in the titles of references should be corrected.\n2. Paper structure improvement:\n - The \"Related Work\" section (Section 5) should be placed earlier (before the “Problem Formulation”) in the paper for a more natural flow.\n3. Typos:\n - In the Appendix B.1, the sentence starting with \"First, ~\" contains an extra closing parenthesis \")\".\n - \"row1,2,3\" → \"rows 1, 2, and 3\"\n - \"Table G\" → \"Table 3\"\n - “Algorithm 3.5” → “Sec. 3.5”", "questions_for_authors": "I have no any questions for the authors.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g6aHvwl3s5", "reviewer": "Reviewer_WSKB", "summary": "This paper presents a diffusion based method for speech separation using microphone arrays. The core idea is to use a DPS based approach where they alternative between diffusion steps, and an estimation of the reverberation/mixing matrix A. The matrix A is necessary to calculate the conditional likelihood updates required by DPS. Results are presented on WSJ synthetic mixtures of audio on different array configurations.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful and encouraging comments.\n>Q: I think the method is very good, but the authors could have done a lot more to strengthen the evaluation. The authors only test with 2 speakers from the WSJ mix, never showing a result with more speakers.\n\nWe sincerely appreciate the reviewer’s kind remarks and will work hard to strengthen the evaluation.\nDuring the rebuttal phase, we ran new experiments on a commonly used 3-speaker reverberant source separation dataset.\nThe results are in our rebuttal to Reviewer Go2n, where we show that ArrayDPS performs surprisingly well. Demos for the 3-speaker separation case are also updated in the demo page **https://arraydps.github.io/ArrayDPSDemo/**.\n\n>Q: Also, I would like to see some real results captured on real-microphone data. All the WSJ mixtures are synthetic, and I am skeptical whether the linear approximation A will hold up extremely well in real-world scenarios where there may be other influences on the captured sound. I would really like to see at least one real-world, non-synthetic example captured on any kind of microphone array.\n\nTo validate our method in real-world conditions, we recorded 15 mixtures in a 7m x 4m x 2.7m room. Two volunteers speak simultaneously under weak environmental noise and the mixtures are recorded by 3 microphones. Demos and recordings are in https://github.com/ArrayDPS/ArrayDPSDemo/tree/main/demo_results_realworld and are updated in the demo site **https://arraydps.github.io/ArrayDPSDemo/**. These new real world results sound consistent with the earlier synthetic results reported in the paper.\n\n>Q: The writing is clear. I do wonder if algorithms 1 and 2 can be simplified to distill the essence of the key high level meaning with more details left in the appendix. Also figure 1 has a lot going on. I don't think all the formulas are needed in the figure, instead, the high level ideas should be sufficient and make it easier to follow.\n\nThanks for your suggestions! Reviewer Go2n also suggests simplifying Algorithm 1 by abstracting out the second-order Heun sampler in EDM [2]. We welcome the advice. For the figures, we will remove some unnecessary formulas to make it clearer.\n\n>Q: Overall, I like the paper, as it feels like it tackles the blind source separation with DPS in an elegant way. I think the main novelty is in the estimation of the A matrix to be used in DPS (otherwise it's just DPS applied to a linear mixture of sources). This could be conveyed a bit better. Also, since that is a big part, it would be nice to see how well the predicted A matrix aligns with some ground truths in synthetically rendered scenarios.\n\nWe are grateful for the reviewer’s encouraging comments and are excited to answer the question related to the predicted A matrix. In the demo page **https://arraydps.github.io/ArrayDPSDemo/**,  we updated a section called **Filter and Source Visualization**, where we show 6 figures corresponding to 6 mixtures.\n\nIn each figure, the first row contains source 1 anechoic source signal $\\tilde{s_1}$, the ground-truth STFT domain RIR filter $h_{1,1}$ (from anechoic source to reference channel), and the reference-channel reverberant source 1 $s_{1,1}$. Of course, the third signal is the convolution of the first two signals.\nThen the second row shows the same for ArrayDPS, i.e., separated virtual source 1 $\\hat{s_1}$, the final FCP estimated filter $\\hat{g}^1_{0\\rightarrow1}$, and the final reference-channel source 1 output $\\hat{s_{1,1}}$.  These two adjacent rows allow direct comparison between ground truth and ArrayDPS.\nSimilarly, row 3 and row 4 show results for source 2.\n\nWe can visually see the difference between the final FCP estimated filter and the ground truth RIR through these figures. Those two signals might not be aligned, but sometimes show similar structure on the spectrogram. On the other hand, the virtual source separated by ArrayDPS is more like the anechoic signal rather than the reverberant signal. We explain this with two reasons:\nFirst, the diffusion model is trained on mostly anechoic clean speech, hence the inclination to generate anechoic sources.\nSecond, FCP performs much better when the input source signal is anechoic (see Appendix G, and our rebuttal to reviewer Go2n, about why we are using the virtual source model). This means outputting an anechoic source (instead of a reverberant one) would satisfy a higher likelihood.\n\nWe also updated our demo site’s 2 speakers separation section with virtual sources. We urge readers to listen to the virtual source samples and compare them with the final output and the ground-truth anechoic source. It appears that ArrayDPS is indeed accomplishing some dereverberation, however, we do not want to over-claim here and intend to verify this promise in future research.\n\n[1] Karras, Tero, et al. “Elucidating the design space of diffusion-based generative models.” Advances in neural information processing systems 35 (2022): 26565-26577.", "claims_and_evidence": "The authors do a good job of presenting evidence to back their claims. Experiment results with numerous ablation studies are presented, and there are qualitative audio examples that let the listener judge for themselves. The show that different array configurations can be used at inference time.", "methods_and_evaluation_criteria": "The authors compare against numerous baselines like TF Gridnet. There are other baselines that could be included (sepformer, demucs, conv tasnet) but since TF Gridnet is state of the art, it suffices. The dataset used, WSJ is also standard and a good choice for comparison.\n\nI think the method is very good, but the authors could have done a lot more to strengthen the evaluation. Firstly, a major advantage of this generative approach is that it can handle any number of speakers during inference. But the authors only test with 2 speakers from the WSJ mix, never showing a result with more speakers. Also, I would like to see some real results captured on real-microphone data. All the WSJ mixtures are synthetic, and I am skeptical whether the linear approximation A will hold up extremely well in real-world scenarios where there may be other influences on the captured sound. Given that the authors are pitching the value of the paper in its generality to different reverbs, mic array geometries etc, I would really like to see at least one real-world, non synthetic example captured on any kind of microphone array.", "theoretical_claims": "The overall theory behind the likelihood estimation and IVA seems sound. Also estimating A in the frequency domain makes a lot of sense.", "experimental_designs_or_analyses": "The experiments are good. The authors evaluate many metrics (SDR, SI-SDR, PESQ, eSTOI) which is nice, and also the WSJ mix is standard. The ablation studies are extremely thorough as well. As I mentioned earlier, I would like to see some real-world examples and examples with more than 2 speakers.", "relation_to_broader_scientific_literature": "This paper does a good job of tying in DPS, a recent and innovative diffusion work, with the problem of multi-channel speech separation. It does a good job of bridging the gap between traditional multi-mic source separation and newer diffusion methods.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The writing is clear. I do wonder if algorithm 1 and 2 can be simplified to distill the essence of the key high level meaning with more details left in the appendix. Also figure 1 has a lot going on. I don't think all the formulas are needed in the figure, instead the high level ideas should be sufficient and make it easier to follow.", "other_comments_or_suggestions": "Overall I like the paper, as it feels like it tackles the blind source separation with DPS in an elegant way. I think the main novelty is in the estimation of the A matrix to be used in DPS (otherwise it's just DPS applied to a linear mixture of sources). This could be conveyed a bit better. Also since that is a big part, it would be nice to see how well the predicted A matrix aligns with some ground truths in synthetically rendered scenarios.", "questions_for_authors": "Have you tried the algorithm with more than 2 speakers? At how many speakers does it fail?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a blind source separation model based on diffusion posterior sampling with an approximated likelihood formulation that models the room acoustics and microphone array geometry. Reviewers unanimously suggested publication, having only raised minor concerns regarding clarity and generalization issues, which in my view do not reduce the value of the paper.
Decision: Accept (poster)