MAPSS: Manifold-based Assessment of Perceptual Source Separation
Authors: Amir Ivry
Samuele Cornell
Shinji Watanabe
Venue: ICLR 2026 Poster (2026)
Keywords: Audio Source Separation
Perceptual Quality Assessment
Uncertainty Quantification
Self‑Supervised Representation
Manifold Learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We introduce two granular measures that quantify interference from competing talkers and distortion of the target signal in audio source separation, along with their error bounds.

Abstract: Objective assessment of audio source‑separation systems still mismatches subjective human perception, especially when interference from competing talkers and distortion of the target signal interact. We introduce Perceptual Separation (PS) and Perceptual Match (PM), a complementary pair of measures that, by design, isolate these leakage and distortion factors.
Our intrusive approach generates a set of fundamental distortions, e.g., clipping, notch filter, and pitch shift from each reference waveform signal in the mixture. Distortions, references, and system outputs from all sources are independently encoded by a pre-trained self-supervised model, then aggregated and embedded with a manifold learning technique called diffusion maps, which aligns Euclidean distances on the manifold with dissimilarities of the encoded waveform representations.
On this manifold, PM captures the self‑distortion of a source by measuring distances from its output to its reference and associated distortions, while PS captures leakage by also accounting for distances from the output to non‑attributed references and distortions.
Both measures are differentiable and operate at a resolution as high as 75 frames per second, allowing granular optimization and analysis.
We further derive, for both measures, frame-level deterministic error radius and non-asymptotic, high-probability confidence intervals.
Experiments on English, Spanish, and music mixtures show that, against 18 widely used measures, the PS and PM are almost always placed first or second in linear and rank correlations with subjective human mean-opinion scores.

Published as a conference paper at ICLR 2026
MAPSS: MANIFOLD-BASED ASSESSMENT AT OF PERCEPTUAL SOURCE SEPARATION
Amir Ivry' Samuele Cornell? Shinji Watanabe²
Electrical and Computer Engineering, Technion Isracl Institute of Technology, Haifa, Israel LLanguage Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA a aivryeieee.ov org
samuele.. corne 11 @ieee..... org swatanabeanddeew cmu. edu
ABSTRACT
Objective assessment of audio source-separation systems still mismatches subjec- tive human perception, especially when interference from competing talkers and distortion of the target signal interact. We introduce Perceptual Separation (PS) and Perceptual Match (PM), a complementary pair measures that, by design, isolate these leakage and distortion factors. Our intrusive approach generates set of fundamental distortions, e.g., clipping, notch filter, and pitch shift from each reference waveform signal in the mixture. Distortions, references, and system outputs from all sources are independently encoded by pre-trained self-supervised model, then aggregated and embedded with a manifold learning technique called diffusion maps, which aligns Eaclidean distances on the manifold with dissimilari- ties of the encoded waveform representations. On this manifold, PM captures the self-distortion of source by measuring distances from its output to its reference and associated distortions, while PS captures leakage by also accounting for dis- tances from the output to non-attributed references and distortions. Both measures are differentiable and operate at a resolution as high as 75 frames per second, allowing granular optimization and analysis. We further derive, for both measures, frame-level deterministic error radius and non-asymptotic, high-probability confi- dence intervals. Experiments on English, Spanish, and music mixtures show that, against 18 widely used measures, the PS and PM are almost always placed first or second in linear and rank correlations with subjective human mean-opinion score
INTRODUCTION
Reliable perceptual evaluation critical for source-separation progress, yet gold-standard listening tests are costly and slow (ITU-T. 1996 2003 2018). Thus, research relies objective metrics that blur two distinct failures, interference from competing talkers and target distortion. Disentangling these modes can better align with listener perception and accelerate trustworthy development.
Existing measures such as the sigmal-to distortion iimaal-oo-distorton ratio (SDR), signal-to-ing -ot-tttrretreeeceeeeeee ratio (SIR). signal-to-artifacts ratio (SAR) (Vincent et al.) (006)), scale-invariant SDR (SI-SDR) (Le Roux et al. 2019) and alike usually compute ratios between source to various disturbances in the waveform domain, offering low complexity and widespread adoption. However, even jointly, they mix leakage and self-distortion into global energy ratios and promote ambiguity to whether an еттог stems for leakage or oif--iitortion. Classical intrusive perceptual and intelligibility metries like the PESQ (Rix et al. 2001). STOI (Taal et al. (011)) and ESTOI (Jensen & Taal 2016) map entire utterance to scaled mean- opinion score (MOS) using hand-crafted auditory features. Designed preliminary for speech enhancement, they perform well for corrupted noiry-eeverberant speech utterances but may not account for leakage, while also lacking to provide access to their inherent granular processing, i..., at the frame level. Learned black-boo metrics such as the DNSMOS family (Reddy al. 2022 that are trained end-to-end to predict crowd-sourced MOS, as well as SpeechBERT (Sacki et al. 2024) and Sheet-SSQA (Huang et al. 2025), ), have shown promising results various speech tasks, but do not offer confidence in their decisions. Spectral-distann metrics are interpretable but tend mask where degradations occur, e.g., the popular Mel-Cepstral Distortion (MCD) (Fukada et al.
Code available
com/ Ami Ivry vory/MAPSSS measures

---

Published as a conference paper at ICLR 2026
1992) collapses the spectral envelope into a global value. Another set of metrics that gained popularity
in recent years include the non-intrusive VQScore built from variational auto encoder (Fu et al. 2024).
the full-reference ViSQOL that uses gammatone "neurograms" (Chinen et al. (200), NORESQ
that learns a relative quality function between two non-matching recordings (Manocha et al. 2021) )
and SCOREQ that is trained estimate utterance-level MOS telephone and synthetic-speent degradations (Ragano et al., 2024). Even when taking into account a broader set of metrics, as
available in recently developed speech quality assessment toolkits (Shi et al., (025), no existing family of measures can simultaneously disentangle leakage from distortion, offer granular analysis, and provide error estimates for their decisions.
We introduce the Perceptual Separation (PS) and Perceptual Match (PM), the first measures for source separation that functionally disentangle leakage and self-distortion. Inspired by auditory theory (Gabrielsson & Sjögren 1979) Jekosch| 2004: Wilson & Fazenda [2014) [Bannister et al. 2024). we apply a set of fundamental distortions to every reference waveform, intended to create a wide cover of perceptual auditory field around the reference. These distortions range from mildly- intrusive short-tailed reverberations to highly degrading hard clipping. A pretrained self-supervised model, e.g., wav2vee 2.0 (Baevski et al. 2020), is used to independently encode the waveforms of references, distortions, and system outputs across all sources, in a resolution as high as 75 frames- per-second. These representations are aggregated and projected via a manifold learning technique called diffusion maps (Coifman & Lafon 2006) onto a low-dimensional manifold. A key property of diffusion maps aligns Euclidean distances between points on the manifold with dissimilarities between their encoded representations. On the manifold, PM quantifies self-distortion by measuring how far an output lies from its attributed reference and the distortions, whereas PS quantifies leakage by comparing these distances with the output proximity to non-attributed references and distortions.
Evaluations on the SEBASS database (Kastner & Herre 2022) with mixtures of English. Spanish, and music, show that compared to 18 widely used measures, PS and PM almost always achieve first- or second-place rankings both linear and rank correlations with human scores, with the exception of Spanish rank correlations, where they remain within the top third. We derive granular theoretical deterministic error radius and high-pros high-probability confidence intervals (CIs) for both measures, enabling frame-level guarantees
the reliability of the measures. In almost all scenarios, the worst-case error radius would not lower the PS and PM rankings. In addition, the normalized mutual information (NMI) (Danon et al. 2005) between the PS and PM values shows that they are highly complementary.
2 PROBLEM FORMULATION
Notational remark. Column vectors and matrices are written in bold and other symbols in non-bole bold.
Consider source separation system performing inference on an audio mixture (Vincent et al. 2018 8 In time frame f that consists of L samples, let N, 2 2 denote the number of active sources and Sy their index set. The observed mixture 2 E Rº is modeled as:
2 Σ Ev. iES,
(1)
For Sy, we denote Yi,f including interference inherent to its original conditions. v, represents system and environmental interference, assumed statistically independent of the sources. The estimation of ys is denoted ý i.
R² the reference signal of the i-th source in frame f. potentially
Given source indices j, Sy in time frame f, our goal is to introduce these two measures:
The perceptual separation (PS) measure quantifies how well ý is perceptually separated from all interfering sources (y,.f biti
The perceptual match (PM) measure quantifies how closely the estimated source ý', perceptually aligns with its reference yi.f-
3 DIFFUSION MAPS: THEORETICAL FOUNDATIONS
Notational remark. Sections are denoted by S. Symbols are carried over from (2) except for indices i that are repurposed, and the subscript f that is dropped since we analyze fixed time frame.
2

---

Published as a conference paper at ICLR 2026
Diffusion maps is manifold learning method that represents high-dimensional data in a low- dimensional space by capturing geometric and structural relationships (Coifman & Lafon 2006). Consider the set X' [x) with x; RM for all e.g., feature vectors from wav2vec 2.0 (Baevski al. 2020 An affinity matrix K NNN is calculated between the high-dimensional vectors:
x -x,,1
Kij = exp
K
where i.j E (1,...,NT N) and V : 0 S Ki S 1, and o = median {
||x;
x5112 i 5). To
account for non-uniform sampling density of points, an a-normalization replaces K by K(a);
K (0) 1J (1;2y)
Kij
where a E [0, and o2 = Σj= Kij- Then, we define the diagonal degree-matrix D given (o)
by ((a) diag (o)²) ..... N-11
N NNN, where of Σド= k and Vi v/()) 0 by
construction. We assume is fixed and for readability we neglect the a notation from now on.
The probability transition operator P K is defined with (3)
D E NNN
(4)
Note P is row-stochastic, so Vi,j : Ps 0 Σ Pu = 1. Spectral decomposition on P reveals
a trivial right eigenvector u, = 1 E IRN with eigenvalue An = Remaining eigenvectors ( (u,) (2-)2-1 W-1 are associated with eigenvalues {>
N-1 and ordered as > \ 2 A2 2 ... 2 AN > 0, so that:
Pu Ae
(5)
Denoting (i) the i-th element the j-th eigenvector, then the embedding of x; onto manifold M can be expressed with the eigenfunctions in (5) by the embedding operation Ve RM N-1,
I (x+) = (>í u (i), (1u2(1),
(6)
where is the number of Markov chain steps, controlling the diffusion scale of the embedding. The eigenvalues in [= are orthonormal under the stationary measure π [TT, .2..... , NN]"
N-1
D T. (0.
E
Let (i.j) be the diffusion distance at time step between two points x, and xy:
N Σ
(Pt
P jm)
Di(i,))
(8)
Tym
m=1
where P denotes the probability of transitioning from node node m in time steps. Intuitively, the diffusion distance measures the similarity between the probability distributions of random walks starting from nodes and j. A key strength of diffusion maps is the equivalence (6)
(i. () (x.) (x(xx))l?:
which is fundamental to approach, it ensures that the Euclidean distances between every two points on the manifold, which we measure in $4.2 and 44.3) align with dissimilarities between their matching gighddimessoonl points, represented by the diffusion distance (8) The embedding in ( is truncated to its first d coordinates and discards the rest. This reduces noise sensitivity and retains the
most meaningful geometric structures (Nadler et al. 2006). The mapping w((!) R. - Rd gives:
(x.) (2/u (f), (1)2(1), Xun(()))
(10)
Consider 1] as the minimal normalized retained sum of the eigenvalues, then d is given by:
Σ de 2 3
d = min k (1......
N}
(11)
ΣN de
3

---

Published as a conference paper at ICLR 2026
Stage
Stage
Stage
Stage
references and system outputs
Perceptual distortions
(vip) g-1
M
Aggregale
Self w-F--serriiieee encodiig
Diffisiion maps embedding
PM and PS calenlations
PM;
Dj
y1
Inputs:
S1
Aggregare
( (60
(xix)
J g
PS
Perceptual distortions
Aggregate
DNr
PM Ny
NN;
PS
Figure 1: Overview of the proposed MAPSS pipeline. Stage 1. Each reference source among the Ny sources in the mixture is independently augmented with bank of perceptual distortions ($4.1). Stage 2. All distorted samples, references, and system outputs across the N₂ sources are aggregated and encoded into self-supervised representations ($4.1). Stage 3. Diffusion maps embed these
representations into a low-dimensional perceptual manifold ($3 $4.1). Stage 4. The PS and PM
measures are computed over this manifold to quantify self-distortion and leakage
4 THE PERCEPTUAL SEPARATION AND PERCEPTUAL MATCH MEASURES
4.1 CONSTRUCTING PERCEPTUAL CLUSTERS THE MANIFOLD
The waveform reference signal of the i-th source, yı, undergoes Np perceptual distortions, e.g., noise gating in different thresholds, vibrato in various rates, and comb filter with several delay-gain pairs. Typically, N₂ E [60, 70). We define the i-th distortion set D; as:
D. = (8.. V.. .
Vp E {1..... N₂F Vi. RR
(12)
with L from [ Each waveform in 2, is independently encoded via pre-trained self-supervised model, e.g., wav2vec 2.0 Baevski et al. 2020). Let + R² - RM be this encoding operator, with M from [3 so X Φ (Ytp)) x; + ly,) X, ① (f.). Applying (12) across all N, sources results in the high high-dimensional set of representations:
X
x x, X,1
N,N, i No).
(13)
with | N, (N, + 2) = N. We define the i-th perceptual cluster cla) on manifold (d (10
cl
d)
(
(xi,p)
1, 1=...,,, N
(14)
where we exclude the embedding of the system output (xt, E Rd 10 from cf since this embedding will be measured against the cluster statistics to produce the PS and PM measures. Including / (&,) in the cluster would create circular dependency that will bias the PS and PM. These distortions were hand-crafted to create a wide perceptual auditory coverage relative to the reference, e.g., by considering mildly-intrusive additive colored noise with signal-to-noise. (SNRs) of 15 dB on one hand, and severely degrading heavy-tailed reverberations on the other hand.
Given Xj E X, the property in guarantees that the Euclidean distance between y (<d) (x:) and v/( (x₃) lowers, so does the diffusion distance between x; and X,. In 44.2 and 4.3 we define our PS and PM measures using Euclidean distances, based our hypothesis that this diffusion distance also aligns with the perceptual alignment between the corresponding waveforms, y: and y,- In 7 we explore if this perceptual -eoomerri hypothesis valid by comparing our measures with human perception.
4.2 THE PERCEPTUAL SEPARATION (PS) MEASURE
For readability, denote the elements of the clusters in 14 sb both here and in 4.3
4

---

Published as a conference paper at ICLR 2026
For source i, we aim quantify the perceptual separation of y, from its non-attributed references by with the Mahalanobis distance (Evans et al. 2023 The empirical centroid and unbiased covariance matrix of the cluster cfa)
are:
Σ
v
Σ bEC
abb
(v-
)
(15)
((d)
cld)
where j E Rdd (dd) E Raddd The squared Mahalanobis distance from the embedding of the i-th
(d)
output 1p
(d) (8:) to c(d) is given by:
( ( )/00) (x) - (1)) (5)0 +etlo)) -1 (x.)) 12. (2)")) (16) Ly V 19
where we use for regularization c = 10 with the d-dimensional identity matrix ((4), We define the measured Mahalanobis distance from L (d) (st,) to its attributed and closest non-attributed clusters as:
1 das (v! (d)
(x:);
(d) :
B((0) das (w/( (x;); (d) {
(17)
(4) (&,) ii 2 Notice that (7 resembles the
with j = arg m
das source permutation minimization processing source separation evaluations (Le Roux et al. 2019).
The measured PS score for y: in the truncated dimension d' is:
A!0) 1/00 - B())
PS (d) =1
PS (d)
10
(18)
where by design 1(t) + 0 > 0 and a higher score is better. Functionally, when A!") < (( then
the i-th output perceptually resembles its cluster members significantly more than competing cluster members and PS." approaches 1. B(a)) < A(()) indicates the opposite, and (s(d) drops towards O.
4.3 THE PERCEPTUAL MATCH (PM) MEASURE
The PM measure aims to quantify how perceptually aligned the estimated output y: is with its reference yt. Let ट(ج) ci v (xx) denote the reference-fiee free i-th cluster. Unlike Equation 15
we compute the unbiased empirical covariance matrix of c( relative to its reference embedding:
(d)
Σ a (xi) (+) v (c) (xx)
Σ
(19)
Then, for p reference in the i-th cluster, is given by d3, [ v (d)
Np). (1,...,Nph the squared Mahalanobis distance from the p-th distortion to its attributed
2 (d)
(xi,));
(xx)).
Eg
following the definition
in Equation 16 Let us define the set of distances:
D (1)
G/:0 -:00{ = dis
v ! (xi,p;
(d) (x.).
p= 1,
N₂
(20)
Empirically, validated that nearly always these distances well-approximated by Gamma distribution, using Colmogorov-Smimon goodness-of-fh tests (Smirnov) 1948 Kolmogorov 1986). The sample mean and unbiased variance of Gi(d) are estimated by:
c
1
Σ gEg(d)
9.
9.00
((d)
Σ
g int=)?
(21)
9
gegi00
and can be moment-matched with Gamma distribution, assuming ( 02 G(
> 0. with parameters:
n2
62 o
R.(4)
(2)
g(d)
(22)
A c
g
5

---

Published as a conference paper at ICLR 2026
Similarly, the squared Mahalanobis distance from the output embedding to its attributed reference 2 (d)
is a!!" -fy
V (d)
(x:);
y (x.),
(d)
Consider 2) = I((kx))I((K) as the regularized
upper incomplete Gamma function of Mathematical Functions, 2024). Then, the PM score for y: in dimension d is:
PM (d) = (
PM;
[0,1],
(23)
where R(C)), 0 are well-defined by design for Np 2 and a higher score is better. If the output a/ lies well within the bulk of its distortion cluster, the Gamma-tail probability is near 1, which may indicate a strong perceptual match. As at") drifts away, the score decays smoothly toward zero, reflecting degradation. When distortions are tightly concentrated and R(()) or 0.4) lower, even small mismatches in à (d) lower PM sharply. As ((()) and 0!(4) grow, the PM tolerates larger a!!)) deviations.
5 ERROR GUARANTEES FOR THE PS AND PM MEASURES
Standing notation and assumptions. We fix frame f with generally N2 2 [ For this proof, consider the specific case of N, 2 with indices i. E Sy. Consider source index and set m = N
d as the dimension of the omitted space in the diffusion maps process, so the embedding notations in the retained, omitted, and complete N - 1-dimensional spaces are respectively y (x,). (x))) (x,) 10 Similarly, clusters c/u)) c(m) and Cj are formed as (m)
(xx):
Σ (u/mm) { ) and (mg Ez), and cross- covariance Cj E Rdxmm We have empirically prevented ill ill-conditioning. as all matrix inversions are Tikhonov-regullriized (Tikhonov& & Arsenin 1977) with e/, where e = =10-6 and / the identity matrix, with context-dependent dimension. When quantify sampling uncertainty, we use dependence- adjusted effective sample sizes via Bartlett method (Bartlett) [946). and sub-Gaussian tails for quadratic forms via the dependent Manson--rigghe inequalities (Adamezak) 2014 Vershynin 2024).
in 441 with means and covariances pairs
Schur decomposition of full versus truncated Mahalanobis distances. For radius error calculations, from 24 to (36,, we assume access to clusters statistics. For the output embedding of source
against cluster j. define statistics aggregate as:
(d)
T (d) (xx)
14j
(d) and (m)
T (mm) (x4)
1 The full cluster
(d)
Σ
(d)
Cj
j4j 4 mm)
C
(m)
(24)
J.j
Block inversion to (16) via the Schur complement (Horn & Johnson| T
2012)
yields:
(V E,)
( (d)
(
(Ta))
(d)
T S ris
(25)
:=b
Ti.j A 1.j C. 1 + {4((4)
(m)
(d)
S,
Σ, (m)
C (2]² + +ella) -1
Cj.
(26)
Since Va, b 2 0: Va - Val S vb (Rudin 1976
Ch. 5), we bound truncation error to 25
18 = (dsr( L (X:);; ): Mij Σÿ) des v
(d) (x;): (d)
(1))
VrT,s,'ra rig.
(27)
PS radius. Let A;, Bi be the full- full-space versions of A B in 17p. Set |ōt,] := IA, A/0h and | : B, /3(2)), with j as in 17).
We empirically confirmed that truncation introduces only mild changes, i.c., 11 |6,..1 << AA B(dd). Thus, first-order Taylor expansion of PS,, the
full-space version of PS." around (A B(a) is valid. Ultimately, we drop quadratic components that were found negligible, and use 1,,l and |0r,5- inside the Taylor expansion, to yield:
B((d) 1 + A(()) |85.3-|
PS; - PS PSS-1s(d1) <
(28)
(A (1) + B(2))² 2

---

Published as a conference paper at ICLR 2026
Combining with (27). the deterministic PS error radius is:
1 rT.,st ri, + A!40) V rr]s- Sy. Fig. (4 + )
(.(d)
Bi
PS;
PS!
(29)
We notice that large residual spread 1 (m))
or cross-block coupling Cj inflate 29 through Sj
PM radius. For source and every distortion index E fI...., Npp. we center cluster coordinates
the reference X;, so A { (xx repeating 24) for the reference- free clusters the d. m, and N i.
( (x,) and A (m)) T (m)
(mm) (Xi.p)
T (m)) (x,). By
1 spaces, we obtain:
Σ
Σ,
(30)
C
T
(m)
where (((0) is defined in 19p.
Exactly as in 25
(26).
we use Schur complement:
Lh (xi.)) (x.). Σx
1.p
(d)
T
(((a) + +c1)
xp
(d)
rin
Tip.
(31)
Tip
A (m) Lp टÍ + cl
(d)
S. {(m) - C Σ d)
+el)
C.
(32)
Let G; be the set of the squared distances in (3 over p and G:(a) its d-dimensional analogue 20
Define per-sample truncation gaps óg.p := ri ripS, S, Tip 2 0 and ómax maxp &G,,p" Employing
elementary algebra and Cauchy-Schwarr Schwarz inequality (Vershynin 2024), we obtain the relations (1
[e.. - (g
=
1
2 σ
N₂ Np
(26max (oo. + o + 6 max)
(33)
P
pml
Again, simple algebra bounds the Gamma matthang parameters
22 with constants C2, C2 > 0:
h4 - k/10 S C1 8max
No
Hg. + Mg!
0, - 0
I C2 8max
N₂
6 + t g
(34)
No
22 o
No
Recalling the distance of the output from its cluster, denoted a
23 we can define using 32
Pat L (x::;; Le (x;), Σ, - dzr
T (c)) (xx) (d)
(x:).
((()
r i,a S. Ti.a
S := 6g.
(35)
In the full space, PM; = Qkk, 0g 10,) (23) Its derivatives with respect to its variables are standard and bounded on compact sets (or Mathematical Functions 2024). Let the truncation ellipsoid B, be such set, so that B, { (k/, 0% a() K/
<!a)) I 6g. 0 (:00) I 8g a; a. (d) < åg
with kg dg.00 denoting the bounds in
344. Since Q increases in & and decreases in a a/0 for the maximum deviation over B, occurs at a corner, and
k, (of Mathematical Functions 2024). the radius can be obtained by:
PM. PM
max (.005
Q(ker a./0.) -(( a. /0!0)
(36)
Dependennce-ddjusted sample size. For any cluster cla)) with ny dependent points, we use Njjeff : ný (1 2 2' Ps.e) with L, = min(f: lpg.el 20.975/ / E} (Bartlett 1946).
PS tail bound. We now resort to the retained d-dimensional space, and estimate cluster statistics
due the finite number of jefe samples. Let (,(0) and S0(00 be the empirical cluster statistics. Vector and matrix- Bernstein and dependent Hanson Hanson-Wright theories (Vershynin) 2024 Props. 2.8. 1, 4.7.
(Adamczak 2014 Thm. 2.5) give for S.'. 8TS E (0, 1/2), with least probabilities 1 /.u 1 0,5)
(1) (d)
5 $12 Xmax
(5(()
(n(2/5PS) //zoettfffff jmns
(37)
2
|>))0) {100
(5)00) rj + ln (2( / Aj5o,
5 C Amax
(38)
7

---

Published as a conference paper at ICLR 2026
with = tracel )/ //max (5)
A first-order perturbation of 4 _B())
17
yields the bounds:
PP (A! 1 22 A! Aiia Amax
(
/dainn
(S((1) + A! A Amax
{ (5)
(39)
r (B.0) S 2 B!!0 Di my
{ Amiin
(1)0 + B!()) D5- / ///axx
{5(()
Amax
(40)
5"
With 4 being the Euclidean gradient norm of PS( at (A! BBld),, for 0.55 -PS + 0,5 (0,
(4) PS. ps(()
S LP V ErS
+ 2PS ( w.p. 2 (1
(41)
PM tail bound. Let R₂ = maxges, 9 and choose confidence levels 5PM 5PM 8PM Concentration bounds for the output quadratic form via Hanson- Props. 2.8.1, 4.7.1) yields
d Manson-Wright Wright (Vershynin 2024
ia E (0. 1/3).
26 ln(2/8PM) 0.00)
3R, In(2/8PM)
|M0,00 -Moturl S
(42)
N
Np
3R In(2 l 212 In (2
PM)
In(2/ ln(2/6,5a)
(00000 - - S
1a |a/0) - a:01| S Riy
(d)
(43)
N,
N,
Np
Just like in 39 and
44 these are mapped to bounds on the parameters (k, 0, a),, yield-
ing Dik: Ai,0 i,a- Consider 8PM 8PM + 5PM + 8PM E (0, 1), then for the local box
1,0
Blue {Kk 13(00)
D 10 0 D.o. la - | A
PM,
(d)
|MM(') <
max
Q(kier (, ,a!") 10(4)] w.p. 2 1 6PPM (44)
6 EXPERIMENTAL SETUP
6.1 DATABASE
We use the Subjective Evaluation of Blind Audio Source Separation (SEBASS) database (Kastner & Herre 2022), a public collection of expertly curated listening tests that aggregates 11, 000 ratings for more than 900 separated signals across five evaluation campaigns. SEBASS covers speech mixtures of 4 male or 4 female speakers, each consisting of English and Spanish pairs. As realistic conversations are monolingual, we separate each mixture into English and Spanish speakers pairs. Also included are music mixtures with drums and without drums, each with sources. Namely, Ny E (2, 3} (I) The split between drum and no-drum mixtures is crucial, as percussion transients create perceptual and algorithmic masking distinct from harmonic content. Each mixture was processed by 32 source separation systems, ranging from classic approaches to deep-learning models. Outputs with 10 s duration, sampled at 16 kHz, were judged by 15 certified raters under the MUSHRA standard (Schoeffler et al.; 2018). which grades output quality between and 100 relative to a reference. These MOS ratings are provided directly by SEBASS and no new human listening tests were conducted by the authors. All experiments in this paper rely solely the existing SEBASS ratings.
6.2 PRE-PROCESSING AND PERFORMANCE EVALUATION
All parameters in this study are selected based on internal properties of diffusion maps and the underlying self-supervised models, or on prior work, and are neither data-driven nor tuned using SEBASS labels. SEBASS provides MOS values at the utterance level. Since our PM and PS measures operate at much finer temporal resolutions, with frame sizes of L = 400 for speech and L = 324 samples for music (I). aggregation from the frame-level to the utterance-level is required to enable comparison with human MOS. PM values are aggregated using a simple average, while PS values are aggregated with prrcettullly-wiighee. scheme inspired by PESQ. For performance evaluation,
8

---

Published as a conference paper at ICLR 2026
Table SRCC and PCC of the PS and PM measures (underlined), their waveform counterparts, and 14 comparative measures, across scenarios. The top-3 results in every column are in bold.
English
Spanish
Music (Drums)
Music (No Drums)
Measure
SRCC
PCC
SRCC
PCC
SRCC
PCC
SRCC
PCC
PS
84.12%
83.74%
82.33%
85.01%
72.87%
77.38%
87.23%
87.81%
84.69%
86.36%
83.41%
85.30%
75.18%
69.88%
88.12%
85.26%
PS (waveform)
73.42%
71.04%
74.69%
75.05%
51.75%
61.83%
78.88%
78.95%
PM (waveform)
69.30%
66.62%
68.27%
67.35%
49.52%
51.77%
74.37%
75.51%
STOI
80.85%
78.40%
78.79%
82.56%
67.29%
71.27%
75.64%
78. 13%
eSTOI
82.14%
82.28%
79.20%
82.68%
54.68%
57.35%
70.06%
74.45%
PESQ
85.56%
84.05%
86.06%
84.98%
61.60%
53.87%
61.26%
60.24%
SI-SDR
78.11%
76.96%
84.07%
81.38%
42.08%
56.98%
70.42%
71.96%
SDR
77.72%
73.13%
84.29%
76.07%
44.78%
54.33%
74.51
75.35%
SIR
51.28%
56.20%
45.67%
55.19%
18.64%
35.76%
51.00%
55.12%
SAR
75.54%
72.98%
78.21% %
73.29%
36.98%
40.81%
66.15%
68.96%
CI-SDR
78.66%
77.41%
84.32%
81.48%
45.02%
55.42%
74.25%
75.11%
DNSMOS-OVRL
63.70%
67.77%
35.34%
43.57%
21.79%
34.27%
13.81%
19.47%
MCD
43.05%
33.86%
45.90%
37.97%
30.27%
42.23%
33.49%
32.19%
SpecchBERTscore
68.58%
67.44%
69.55%
70.48%
52.33%
59.71%
75.60%
81.13%
Sheet-SSQA
41.17%
51.38%
61.06%
73.01%
39.40%
29.03%
14.19%
5.17%
UTMOS
55.53%
55.43%
52.22%
55.75%
-9.24%
-8.25%
12.59%
7.72%
NISQA
60.78%
67.62%
63.37%
66.58%
27.27%
41.73%
42.33%
48.07%
SCOREQ
77.02%
81.85%
82.32%
83.13%
62.73%
68.56%
75.35%
75.84%
NORESQA
55.82%
41.53%
58.56%
47.24%
-2.34%
-0.80%
61.21%
65.45%
VQScore
36.08%
37.31%
36.93%
40.65%
9.75%
-7.68%
31.14%
33.83%
ViSQOL 72.59%
74.55%
76.48%
75.48%
71.94%
65.48%
78.75%
70.31%
correlate the aggregated PM and PS values with the utterance-level MOS values using the Pearson product-moment correlation coefficient (PCC) (Benesty et al. 2009) and the Spearman rank-order
correlation coefficient (SRCC) (Sedgwick 2014). We set a
(3) to eliminate density-dependeen
bias from the embedding, and The retained dimension d is in [20, 40) 100 using r 0.99 11). ) as done on (Fjellström & Nyström
(6) keep the diffusion operator focused local structures.
2022). Although diffusion maps and Laplacian Eigenmaps (L.)) Belkin & Niyogi
(0yyogi(003) (2003) spectrally related, our setting with a = and / does not reduce to I.H. The parameter a = introduces a density-normalized kernel that yields operator approximating intrinsic Laplace- Laplace Laplace-Belrrami geometry rather than the sampling density Coifman & Lafon (200)). Furthermore, diffusion maps embed points via eigenvalue- -eiighhee coordinates, which is essential for preserving the diffusion distance in (9). whereas LE uses unweighted eigenvectors of graph Laplacian.
EXPERIMENTAL RESULTS
Results are from zero-shot SEBASS inference, without training or data-driven parameter tuning.
Table benchmarks the proposed PS and PM measures against 14 widely-used metrics for audio quality and also versus its waveform-only version, denoted PS (waveform) and PM (waveform). In this variant, the waveforms are passed directly through the diffusion-maps process, skipping the self-supervised representations. This allows us to isolate and quantify the contribution of self- supervised embeddings the effectiveness of the PS and PM measures. For speech, we used
wav2vec 2.0-based (Baeyski et al.) (2020) model with features of dimension M = 1024 ($3) and 24 transformer layers, and for music we use the MERT model (Li et al. 2024) with M 768 and 12 transformer layers. Previous work has shown that earlier layers of self seperviee speech models are often more perceptually stable (Pasad et al. 202 2023). our experiments, layers 1-3 produce nearly identical performance, with correlation coefficients that differ by less than 1% absolute, as shown in Figure in Appendix C For concreteness, we illustrate results using layer 2 for speech, layer with drums, and layer 3 without drums. Our conclusions hold uniformly across these layers. PS and PM consistently achieve top PCC values, aside from minor advantages by PESQ and STOL. For SRCC, measures dominate in music, but trail PESQ in English and SDR-based metries in

---

Published as a conference paper at ICLR 2026
NMI (PS Threshold)
NMI (PM Threshold)
English
0.15-
Spanish
NMI
Drums
11 00
No drums
0.05
0.2
0.4
1.0
0.6
PS Threshold
PM Threshold
Figure 2: NMI between the PS and PM measures across their thresholded values.
Table 2: Deterministic error radius and probabilistic 95% CI of the SRCC and PCC across scenarios.
English
Spanish
Music (Drums)
Music (No Drums)
Measure
SRCC
PCC
SRCC
PCC
SRCC
PCC
SRCC
PCC
PS radius
0.16%
0.21%
0.10%
0.14%
0.40%
0.72%
0.14%
0.11%
PS CIs (95%)
30.03%
10.29%
26.39%
8.85%
28.71%
12.21%
12.69%
4.11%
PM radius
0.11%
0.99%
0.18%
1.23%
0.29%
1.39%
0.02%
1.04%
PM Cls (95%)
7.23%
3.83%
8.98%
4.28%
6.25%
4.15%
4.75%
1.77%
Spanish. These results position the PS and PM as valid measures for leakage and self-distortion for source separation systems. Encoding proves essential, as waveform-onl variants perform worse. Finally, PS and PM outperform SpeechBERTScor,, showing the benefit of diffusion maps over cosine similarity.
We examine the complementary relationship between the PS and PM using NMI (Danon et al. 2005) which captures statistical dependence beyond linear effects, with lower NMI indicating less shared information. Each measure is normalized per utterance to [0. 1]. For thresholds {0.1, 0.2, ..... 1), we retain frames with PS below the threshold and compute the NMI between aligned PSSPM pairs. The procedure is repeated with thresholding on the PM. Figure 2 shows the NMI decreases toward zero as complementary when separation
thresholds tighten, suggesting that the PS and PM become quality is poor. At the loosest thresholds, NMI rises up to 0. 15, yet full redundancy corresponds to 1. These results support reporting both PS and PM, as each captures failure modes missed by the other.
Frame-level error bounds of the measures were derived in Table 2 presents their propagation into PCC and SRCC error bounds. The error radius never exceeds 1.39%, a bias that rarely affects the performance ranking in Table The 95% CIs highlight that the PS carries higher statistical uncertainty, whereas the PM is statistically more robust. This positions the PS as a complementary diagnostic, capturing perceptual leakage that the PM misses, at the cost of greater variability.
Additional experiments that are presented in the Appendices include analyzing the PCC and SRCC error bounds of the measures under different self-supervised models, introducing failing points of the PS and PM relative PESQ as our strongest competitor, testing the generalization of our measures under out of distribution distortions, demonstrating how the PS and PM each disentangles leakage and self-distortion in audio source separation and emphasizing specifically their advantage relative to the SDR family of measures, and examining our method under multilingual information of both English and Spanish.
CONCLUSIONS
We introduced the PS and PM, frame-level measures that showed competitive correlations with human MOS for source separation evaluation by operating on diffusion map embeddings of self-supervised audio representations. We derived deterministic truncation bias and non-asymptotic CIs for both measures, making scores interpretable under quantified uncertainty. Looking forward, PS and PM can serve as diagnostic tools to localize whether errors stem from target distortion or cross-talk, while their differentiability enables separation under confidence monitoring. Finally, their uncertainty bounds offer a principled layer for benchmarking, supporting fairer hyper-parameter sweeps and reporting standards.
loss terms curriculum triggers to balance fidelity and
10

---

Published as a conference paper at ICLR 2026
Ethics Statement. This work does not involve human subjects personally identifiable information. We use only existing datasets under their respective licenses and terms use, and we do not redistribute any data where licenses restrict sharing. Our study complies with the ICLR Code of Ethics. We assessed foreseeable risks and did not identify specific, material harms arising from the methods or results presented here. We will release implementation details and seripts to support responsible reuse and verification.
Reproducibility Statement. We provide complete code as anonymous supplementary material in a separate zip file. It contains the complete inference pipeline, including the frame-level calculation of the PS and PM measures and their determinstic and probabilistic error bounds.

---

Published as a conference paper at ICLR 2026
REFERENCES
Radoslaw Adamczak. A note on the hanson-wright inequality for random vectors with depen- dencies. Electronic Communications in Probability, 20:1-13, 2014. URL https://apt
ietrantisscholee www.ccoaaaa.rg/orrrrrroee.e Corp//Crrusscc ID: TD:::196770 111677390)
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vee 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33: 13:11249-1222 124600, 2020.
Scott Bannister, Alinka E. Greasley, Trevor J. Cox, Michael A. Akeroyd, Jon Barker, Bruno Fazenda, Jennifer Firth, Simone N. Graetzer, Gerardo Roa Dabike, Rebecca R. Vos, et al. Muddy, muddled, or muffled? understanding the perception of audio quality in music by hearing aid users. Frontiers in Psychology. 15, 2024.
Maurice S. Bartlett. On the theoretical specification and sampling properties of autocorrelated time-series. Supplement to the Journal of the Royal Statistical Society, 8(1):27-41, 1946.
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionaliity reduction and data representation. Neural Comput., 15(6):1373-1336, June 2003. ISSN 0899-7667. doi: 10.1162/ 0899766033211803177 URL
Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. In Noise reduction in speech processing, pp. 1-4. Springer, 2009.
Jean Carletta, Simone Ashby, Sehastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al. The AMI meeting corpus: A pre-announcement. In Proc. International workshop on machine learning for multimodal interaction, pp. 28 39. Springer, 2005.
Sanyuan Chen, Chengyi Wang. Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, and Furu Wei. WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 1656)1150---518, 2022. doi: 10. D0.110///SSSP.222..18877
Michael Chinen, Felicia SC Lim, Jan Skoglund, Nikita Gureev, Feargus O' O'Gorman, and Andrew Hines. ViSQOL v3: An open source production ready objective speech and audio metric. In twelfth international conference on quality of multimedia experience (QoMEX), pp. 1 1-6. IEEE, 2020.
Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and computational harmonic analysis, 21(1):5-30, 2006.
Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli, Unsupervised cross lingsll representation learning for speech recognition. In Proc. Interspeech, pp. 2426-2430, 2021.
Leon Danon, Albert Díaz-Guilera Jordi Duch, and Alex Arenas. Comparing community structure identification. Journal of statistical mechanics: Theory and experiment, 2005(00) PP90000 2005.
European Broadcasting Union (EBU). EBU recommendation R128: Loudness normalisation and permitted maximum level of audio signals. Technical recommendation, European Broadcasting Union, Geneva, Switzerland, 2011. 1. URL //tech. ebu. ch/ htt///dcs////2/ påf
Luke Evans, Maria K Cameron, and Pratyush Tiwary. Computing committors in collective variables via mahalanobis diffusion maps. Applied and Computational Harmonic Analysis, 64:62-101, 2023.
Carmina Fjellström and Kaj Nyström. Deep learning, stochastic gradient descent and diffusion maps. Journal of Computational Mathematics and Data Science, 4(1), 2022.
Szu- Rei-Wei Fu, Kuo-Hsuan Hung, Yu Tsao, and Yu-Che Yu-Chiang Frank Wang. Self-supervised speech quality estimation and enhancement using only clean speech. International Conference on Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Toshiaki Fukada, Keiichi Tokuda, Takao Kobayashi, and Satoshi Imai. An adaptive algorithm for mel-cepstral analysis of speech. In Proc. ICASSP, volume 92, pp. 137-140, 1992.
Alf Gabrielsson and Häkan Sjögren. Perceived sound quality of sound houndrepproducnng systems. The Journal of the Acoustical Society of America, 65(4):1019-10) 1033, 1979.
Matthias Hein, Jean- Yves Audibert, and Ulrike Von Luxburg. From graphs saaffolds-waal and strong pointwise consistency of graph laplacians. International Conference on Computational Learning Theory, pp. 470-485. Springer, 2005.
Roger A Hom and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. /EEF/ACM Transactions Audio, Speech, and Language Processing, 29:3451-3460, 2021.
doi: 10.1 (A0.///TAASS..2222112222.
Wen-Chin Huang, Erica Cooper, and Tomoki Toda. SHEET: Multi-purposee Open-source Speech Human Evaluation Estimation Toolkit. In Proc. Interspeech, pp. 2355-2359, 2025.
Kuo-Hsuan Hung, Hsin-Yi i Lin, Cheng-Yu Li, Yu-Tseng Tsai, Chia-Ping Chen, and Hsin-Min Wang. Boosting Self-Supervised Embeddings for Speech Enhancement. In Proc. Interspeech, pp. 5340-5344, 2022. doi: 10.21437/mmeesppecc..000 2022- 10255.
ITU-T. IU-T P.800: Methods for subjective determination of transmission quality. Technical re- port, Inter. Telccommunication Union, 1996. 800/en
wtw. wttp://tttttttteetteeeeeeete
ITU-T. ITU-T P.835: Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm. Technical report, Inter. Telccommunication Union, 2003, URL https::/ itu.
ITU-T. ITU-T P.862.2: Wideband extension to Recommendation P.862 for the assessment of wide- band telephone networks and speech codees. Recommendation, International Telecommunication Union, Telecommunication Standardization Sector (ITU-T), Geneva, Switzerland, November 2007.
int URL [ttppstss://aaadee. 1tt.:u.nnn1111..11111111... 10000/11..1000000//0300101111110
ITU-T. ITUT P.808: Subjective evaluation of speech quality with crowdsourcing approach. Technical report, Inter. Telecommunication Union, 2018. URL https: itu. int/
Fec/T-REC- P.0eee
Ute Jekosch. Basic concepts and terms of "quality", reconsidered the context of product-sound quality. Acta Acustica united with Acustica, 9066):999-1006, 2004.
Jesper Jensen and Cees H. Taal. An algorithm for predicting the intelligibility of speech masked by modulated noise maskers. IEEE/ACM Transactions Audio, Speech, and Language Processing, 24(11):2009-2022, 2016.
doi: 10.100//AALLP22012555577
Thorsten Kastner and Jürgen Herre. The SEBASS-DB. A consolidated public data base of listening test results for perceptual evaluation BSS quality measures. Proc. International Workshop on Acoustic Signal Enhancement (/WAENC). IEEE, 2022.
Maurice Kendall and Jean D. Gibbons. Rank Correlation Methods. Edward Arnold, London, edition, 1990.
Vahid Khanagha, Dimitrios Koutsaidis, Kshitij Kalgaonkar, and Sridha Srinivasan. Interference Aware Training Target for DNN-based Joint Acoustic Echo Cancellation and Noise Suppression. In Proc. Interspeech. ISCA, 2024. URL https: ///www isca- archive. wtw.chhiieeerroenneeeeeeee..eee. 2024 2024/Khanagha22 interspeech pdf
Andrey N. Kolmogorov. On the empirical determination of distribution law. In Albert N. Shiryaev (ed.), Selected Works of A. N. Kolmogorov, Volume II, pp. 139-146. Springer, New York, 1986.
13

---

Published as a conference paper at ICLR 2026
Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R. Hershey, EDR-half-Bair or Well Done? In Proc. ICASSP. pp. 626-630, 2019.
Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao MA, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, zili wang, Yike Guo, and Jie Fu. Mert: Acoustic music understanding model with large-scale self-supervised train- ing- In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, pp. 12181-12204, 2024. URL https:ttppoccceepiniiiieee 3
wctpppp....ppppppeepeepppppepeee foreffilesse wttp:eepeeeeeeeeeeeeeeeeeap /ttee///pper/222//LLL. Papeer-pp--- Conference..pd.
Pranay Manocha, Buye Xu, and Anurag Kumar. NORESQA: A framework for speech quality assessment using non-matching references. Advances in neural information processing systems, 34:22363-22378, 2021.
Omer Moussa and Mariya Toneva. Brain-tuned speech models better reflect speech processing stages in the brain. In Proc. Interspeech, pp. 2905-2909, 2025.
Boaz Nadler, Stéphane Lafon, Ronald R Coifman, and Ioannis Kevrekidis. Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Applied and Computational Harmonic Analysis, (1(1):11331111 2006.
Digital Library of Mathematical Functions. Regularized incomplete gamma functions. //dlmf www.:iiistttto....omm 2024.
http
Ankita Pasad, Ju-Chieh Chou, and Karen Liveseu. Layer-wise analysis of supersupd speech representation model. In Proc. Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 914-921. IEEE, 2021.
Ankita Pasad, Bowen Shi, and Karen Livescu. Comparative layer-wise analysis of self-supervised speech models. In Proc. ICASSP. IEEE, 2023.
Alessandro Ragano, Jan Skoglund, and Andrew Hines. SCOREQ: Speech quality assessment with contrastive regression. Advances in Neural Information Processing Systems, 37:105702-1057299 2024.
Chandan KA Reddy, Vishak Gopal, and Ross Cutler. Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In Proc. ICASSP, pp. 886-890. IEEE, 2022.
Anthony W. Rix, John G. Beerends, Michael Hollier, and Andries P. Hekstra. Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs. In Proc. ICASSP, volume 2, pp. 749-752, 2001.
Walter Rudin. Principles of Mathematical Analysis. International Series in Pure and Applied Mathematics. McGraw-Hill, New York, 3 edition, 1976.
Takaaki Saeki, Soumi Maiti, Shinnosuke Takamichi, Shinji Watanabe, and Hiroshi Saruwatari. SpeechBERTScore: Reference-aware automatic evaluation of speech generation leveraging NLP evaluation metrics. In Proc. Interspeech, pp. 4943-4947, 2024. doi: 10.214377nntesspeechh 2024-1508.
Michael Schoeffler, Sarah Bartoschek, Fabian-Robert Stoter, Marlene Roess, Susanne Westphal, Bernd Edler, and Jürgen Herre. WebMUSHRA a comprehensive framework for web-based listening tests. Journal of Open Research Software, 6(1), 2018.
Manfred R. Schroeder. New method of measuring reverberation time. Journal of the Acoustical Society of America, 37(2):009442,, 1965. doi: 10.1121/1.19093.1
Philip Sedgwick. Spearman's 's rank correlation coefficient. BMJ, 349:g7327, November 28 2014. doi: 10.1 1366/mj.g7733
14

---

Published as a conference paper at ICLR 2026
Jiatong Shi, Hye-jin Shim, Jinchuan Tian, Siddhant Arora, Haibin Wu, Darius Petermann, Jia Qi Yip. You Zhang, Yuxun Tang, Wangyou Zhang, et al. VERSA: A versatile evaluation toolkit for speech, audio, and music. In Proceedings of the 2025 Conference the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations), pp. 191-209, 2025.
Nikolai V. Smimov. Table for estimating the goodness of fit of empirical distributions. The Annals of Mathematical Statistics, 19(2):77-2281, 1948. doi:
Cees H. Taal, Richard C. Hendriks, Richard Heusdens, and Jesper Jensen. An algorithm for intel- ligibility prediction time-frequency weighted noisy speech. IEEE Trans. Audio, Speech and Language Processing, 19(7):212522133 2011.
Robert J Tibshirani and Bradley Efron. An introduction to the bootstrap. Monographs statistics and applied probability, 57(1):1-436, 1993.
Andrei N. Tikhonov and Vasiliy Y. Arsenin. Solutions of Ill-posed Problems. Winston & Sons, Washington, DC, 1977.
Aditya Vaidya, Evelina Fedorenko, and Josh McDermott. Self-supervised models of audio effectively explain human cortical responses to speech. In International Conference Machine Learning (ICML), pp. 21820-21838. PMLR, 2022.
Roman Vershynin. High- Ligh-Dimensiona Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2nd edition, 2024.
Emmanuel Vincent, R.émi Gribonval, and Cédric Févotte. Performance measurement in blind audio source separation. IEEE Trans. Audio, Speech and Language Processing, 14(4):1462-146, 2006.
Emmanuel Vincent, Tuomas Virtanen, and Sharon Gannot (eds.). Audio Source Separation and Speech Enhancement. John Wiley & Sons, 2018.
Alon Vinnikov, Amir Ivry, Aviv Hurvitz, Igor Abramovski, Sharon Koubi, Ilya Gurvich, Shai Pe'er Xiong Xiao, Benjamin Martinez Elizalde, Naoyuki Kanda, Xiaofei Wang, Shalev Shaer, Stav Yagev, Yossi Asher, Sunit Sivasankaran, Yifan Gong, Min Tang, Huaming Wang, and Eyal Krupka. The NOTSOFAR- challenge: New datasets, baseline, and tasks for distant meeting transcription. In Proc. Interspeech, pp. 5003-5007, Kos Island, Greece, 2024. doi: 10.21437/mmtesspeechh 2024- 1788.
Alex Wilson and Bruno Fazenda. Characterisation of distortion profiles in relation to audio quality. In Proc. of the 17th Int. Conference on Digital Audio Effects (DAFx-14), pp. 1-8, 2014.
Neil Zeghidour, Olivier Teboul, Félix de Chaumont Quitry, and Marco Tagliasacchi. LEAF: learn able frontend for audio classification. In International Conference on Learning Representations (ICLR), 2021. URL http:ttps/// httptttts///peneeee/ee http////pppw/.ee.c//eep net for forum? 1dc.mmm261djM0766b6666 6BCb63 9m
15

---

Published as a conference paper at ICLR 2026
A PERCEPTUAL DISTORTIONS APPLIED IN THE PS AND PM CALCULATIONS
Table 3: Distortions applied to the references when calculating the PS and PM measures fs is the sampling frequency, and Ays and ARMS mark the 3Sth-percentil and RMS absolute amplitudes.
Distortion
PS
PM
Notch Filter
Center frequencies:
Number of notches:
500, 1000, 2000, 4000, 8000 Hz
< 20
Operating band:
80 Hz 0.45fs
Notch spacing:
2 300 Hz
Bandwidth:
# Hz
Comb Filter
Delay:
Delay-gain pairs: (2.5 ms, 0.4), ms, 0.5).
2.5-15 ms Feedback gain:
(7.5 ms, 0.6), (10 ms, 0.7),
0.4-0.9
(12.5 ms, 0.9)
Tremolo
Rate:
Rate:
1, 2, 4, 6 Hz
1, 4, Hz
Depth:
Depth:
0.3-1.0
Additive Noise
SNR:
SNR:
-10 -5, 0, 10, 15 dB
-15, -10. 0, 5, 10, 15 dB
Noise color: white, pink, brown
Noise color: white, pink, brown
Additive Tone
Harmonic
Tone frequency: 100, 500, 1000, 4000 Hz Amplitude:
Tone frequency:
100, 500, 1000, 4000 Hz Amplitude: {0.4, 0.8, ARMS
0.02-0.08 (absolute)
Reverberation
RT (Schroeder 1965):
Exponential tail length: 50. 100, 200, 400 ms
0.3-1.
Early tail length:
Decay scaling: 0.3, 0.5, 0.7, 0.9.0.9
5, 10, 15, 20 ms
Noise Gate
Threshold:
Threshold:
0.005, 0.01, 0.02, 0.04 (absolute)
(0.05, 0.1, 0.2) x Ags
Pitch Shift
Offisets:
Offsets:
-4 22 +2, +4 semitones
4 -2, +2, +4 semitones
Low-Pass Filter
Cutoff:
Cutoff rule:
2000, 3000, 4000, 6000 Hz
spectral-energy quintiles:
50, 70, 85, 95%
Rounding:
nearest 100 Hz
High-Pass Filter
Cutoff:
Cutoff rule:
100, 300, 500, 800 Hz
spectral-energy quintiles:
5, 15, 30, 50%
Rounding:
nearest 100 Hz
Echo
Delay:
Delay: 50, 100, 150 ms
5-20 ms
Gain:
Gain: 0.4, 0.5, 0.7
0.3-0.7
Hard Clipping
Threshold:
Threshold: {0.3, 0.5, 0.73 Ags
0.3, 0.5, 0.7 (absolute)
Vibrato
Rate:
Rate:
5, Hz
3, 7 Hz
Depth:
Depth:
0.001-0.003 (fractional stretch)
adaptive, clipped to 0.01-0.05
16

---

Published as a conference paper at ICLR 2026
PS over (A(d) B(())
over (a/ ky at 8
3-
2-
2
-0.4
1
0.2
0.2
0-
0.0
0.0
2
3
4
5
1
a
PM over (8/") _k(0)) at a/
PM over (a/") 0/00)
at k/a0
5
4-
E
E
1
0.0
2
3
4
5
3
4
Figure 3: Functional behavior of the PS measure with 0.25, 0.5, 0.75 contour lines, and of the PM measure in three different setups of 0!4) 1, (d) 1, [d)
1.
B ADDITIONAL EXPREIMENTAL SETUP DETAILS
B. THE PS AND PM MEASURES
The functionality of the measures demonstrated in Figure 3 and illustrates the bohavior explained in 4.2 and
The empirical distributions of the frame-level values of the measures are shown Figure The PM and PS metrics exhibit contrasting distribution patterns. PM values cluster predominantly around zero with minimal density near one, while PS concentrate near one with virtually no occurrence near zero. Although frame-level human speech quality ratings comparison, these patterns raise comparisons to how humans might perceive audio disturbances. The PM distribution aligns intuitively with human perception, as listeners typically penalize speech quality severely when disturbances occur, making ratings near the scale minimum unsurprising, However, real granular human ratings would likely show less extreme clustering around zero due to perceptual and rating scale complexities. The PS behavior presents more complex interpretative challenge. Previous research suggests that humans perceive leakage as more -degrddng than self-distortions, particularly in acoustic echo cancellation contexts (Khanagha et al. (024), yet our findings here do not support this hypothesis. Whether this discrepancy stems from dataset characteristics, limitations of the PS measure itself, or the mismatch between granular PS values and aggregated human ratings remains unclear and warrants future investigation beyond the scope of this study.
publicly available for direct
17

---

Published as a conference paper at ICLR 2026
Speech (English and Spanish)
Speech (English and Spanish)
17.5
15.0
12.5
Density (%)
10.0
Density
2.5
0.0
Music (Drums and No Drums)
Music (Drums and No Drums)
35
30
25
Density (%)
Density
20
15
10
5
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
PM
P5
English
Spanish
Music (drums)
Music (no drums)
Figure The distribution of and PS values across speech and music scenarios from the SEBASS database.
B.2 THE SEBASS DATABASE
The SEBASS dataset suits this study for several reasons. Multilingual coverage of English and Spanish validates sanguage-agnostee behavior, while music tests robustness highly transient material. Large algorithmic spread creates rich output clusters that stress-test our methodology, and the dense sampling of raters allows for more reliable estimation of the true mean-opinion score of subjective human opinion. Figure 5 shows that the speech reference signals have been recorded in relatively clean environment with SNRs between 3.9 dB to 41.7 dB, with an average of 25 dB.
B.3 PRE-PROCESSING
We recognize that English and Spanish speakers rarely participate in the same conversation in real-life scenarios. To emulate realistic scenarios, we separate each 4-speaker mixture into their English and Spanish speakers, creating for each language two mixtures where the one has pair of male speakers and the other a pair female speakers. We acknowledge the uncertainty this step induces, as residuals of English may be present in the output signal of a Spanish speaker, and vice versa. It should be mentioned that listening tests have rendered this cross-language leakage extremely negligible. This may be since, as expected, source separation systems are able to leverage languages meaningful feature to recognize leakage and remove
Every waveform, including references, distortions, and outputs from all sources of the mixture, undergoes independent loudness normalization. We use the EBU Recommendation R- R-128 (European) Broadcasting Union (EBU) 2011) and set the target level of each waveform to loudness units relative to full scale (LUFS) of -23. the peak magnitude of the scaled waveform exceeds one, we attenuate it avoid digital clipping. This step removes loudness bias, known wrongly affect both human
18

---

Published as a conference paper at ICLR 2026
SNRs of Reference Sources English
SNRs of Reference Sources Spanish
3.5
3.0
Density (%)
Density
1.5
1.0
0.5
0.0
10
20
30
40
10
20
30
40
SNR [dB]
SNR [dB]
Figure 5: Frame-level SNR estimations for English and Spanish references in the SEBASS database.
and algorithmic quality judgments, while preserving inter-speaker level relations across the outputs. Since the PS and PM measures address source separation, also filter out any frames in which there are not at least two active sources using energy-based -based thresholding.
When applying diffusion maps, embedding. This choice ensures that the PS and PM measures reflect the intrinsic geometric structure of the manifold rather than sampling density variations, which would introduce instead artificial distortions into the representation. We set = in (6). to keep the diffusion operator focused on local neighborhoods and not being blurred by multi-step mixing.
in eliminate density-depentee bias from the
B.4 FRAME-LEVEL UTTERANCE- AGGREGATION THE MEASURES
At trial I, let us denote the PM value of the i-th output of source-separation system g in time frame PM.. Let F holds the time-frame indices with at least two active sources, and let Fi F be its subset of time-frame indices in which the i-th source is active. Then, the utterance-level PM measure after average aggregation is given by:
PM utt
PM JEF!
(45)
F
Although average aggregation assumes that human listeners perceive global audio quality by weighing local events equally, which is evidently not the (Rix et al.) 2001), we chose carry it for the PM since its behavior already exhibits strong and frequent granular penalties where the score drops around zero. Thus, is assumed that standard human behavior that weighs negative experience heavily in the utterance-level score is implicitly carried out by the nature of the PM measure itself.
However, this is not the case for the PS measure. Here, the aggregation window-bate based pooling and logistic mapping used inside PESQ (Rix et al. (001). Again, considering only time frame indices in Fi and dropping the rest, slides across the PS measure with hop size of H frames. Using the p-norm, we define the following:
applied inspired by the
consider a window of size W frames that
1/p
W
pd m.
1
P i.
(46)
W
1)
w=1
where E (1,....
(....,M)) MD and M. is the number of possible windows:
F
W
))
M max
1,
(47)
H
19

---

Published as a conference paper at ICLR 2026
We then calculate the following root mean square expression:
M
Σ
(48)
M
and eventually the aggregated PS measure is given by:
PS jut
0.999
+ 1 + cxp (xxp01133669 ga. + 3.8224)
(49)
where the constants were chosen according to (ITU-T) 2007). Here, we penalize lower scores explicitly using the p-norm better match human perceptual aggregation.
B.5 CORRELATION COEFFICIENTS BETWEEN AGGREGATED MEASURES AND MOS
At trial I, let the utterance-level MOS of the i-th output from separation system q be ufth Given Q independent source separation systems such that q E (1.....04 consider the Q-dimensional vectors:
PS!
PPS L,u
PSQ.1 imu!
(50)
jut
PM .unt
PM Luil
PM Lull
T
(51)
T
(52)
The PCC
(Benesty al. 2009) measured twice, for the PS and the PM, as follows:
PS
peed
PS Suurs v!)
(53)
PS
s,unt
2
PM M.ll .utt
v
peed
(PM um v!)
(54)
PM
v
juit
where PS PM and V are the centered versions of PS mum
PM and v/. respectively.
Let R RQ__ > R the ranking operator, which the presence of ties assigns the average ranks.
The SRCC (Sedgwick)
2014) is measured for the PS and the PM:
Pi
Stree (PS/ v.) pecc (R PS (PS R (v!))
(55)
stee./ Pt
(PM v!) poo.t (R [PM R (v!)
(56)
We report these correlation coefficients per English, Spanish, and music mixtures scenarios separately, Let us denote N/ the number of active sources in trial during frame f. Then, given a scenario with we mark the maximal number of sources in trail with
L independent trials such that E {l...., L}. I....,L).
No N
N max
= max N
(57)
SEF
20

---

Published as a conference paper at ICLR 2026
Then, for the PS and PM measures, the PCC and SRCC report per scenario given by:
N
PSnE
pec,
(PS
m v.).
(58)
21=1 max [=1
No
Nus
PM
peel
PMM v!)
(59)
E1=1 No
1=1
N
PS
Ps
soce.l
PS
uun v.).
(60)
ΣE
No
max
i=
a=1
N
aM
Ps
soc,.
(PM
iuur: v!)
(61)
NI
max
1=1
C ADDITIONAL EXPERIMENTAL RESULTS
Table 4: Self-supervised architectures, their pre-trained checkpoints, scenarios, and number of
transformer layers.
Architecture
Checkpoint
Scenario
Transformer Layers
WavLM Large
microsoft/wavlm--laree
English
24
WavLM Base
microsoft/wavlm--asse
English
12
wav2vec 2.0 Large
facebook/wwa/2vee2-lrrgg la6ge-1660
English
24
wav2vee 2.0 Base
facebook//wav2vee2-
English
HuBERT Large
facebookkhuberr-llgg--l0kk
English
24
HuBERT Base
facebook//huberr-bbse--kk//
English
wav2vec 2.0 Large
Spanish
24
MERT
m-a-P/MERT-v1-995M
Music
12
We begin by analyzing how performance depends on the choice of the pre-trained self-supervised model, the purpose of which is encoding waveforms into perceptual representations before they are fed into the diffusion maps. Table 4 lists the models we examine in this study. We consider six different models for English mixtures, based on the wav2vec 2.0 (Baevski et al.) (020)) WavLM (Chen) al., 2022), and HuBERT (Hsu et al., 2021) backbones, with Figure [ demonstrating their layer- wise performance. When using "Large" versions of the models, for both PCC and SRCC values, earlier layers frequently produce representations that allow superior results that gradually decline toward deeper layers, showing approximately 10% average absolute degradation between extremes. Existing layer- wise analysis already reported that acoustic and phonetic content is richly represented in intermediate layers, while deeper layers shift toward semantic abstraction (Pasad et al., 2023 Vaidya et al., 2022). Additional work confirms that distortion sensitivity peaks the lower or middle layers and diminishes in deeper ones (Hung al.) (002). and that pretrained models tend to lose low-level signal fidelity in their deepest layers (Moussa & Toneva, (025). A notable data point appears in the final layers of wav2vec 2.0 with sharp drop performance, especially for PS. This is likely due to its contrastive learning pretraining objective, which drives later layers to specialize in predicting quantized latent codes rather than preserving acoustic detail. For the "Base" versions of the models, we observe a somewhat different behavior. At low and middle layers, their performance often quite competitive with the "Large" variants, and in several cases the former even outperforms the latter in deeper layers. However, for WavLM, the gap widens toward the final layers, with the "Large" version consistently outperforming. Interestingly, wav2vee 2.0 Base does not exhibit the sharp degradation observed in its counterpart and instead its deeper layers remain stable and even show improvements for PS, suggesting that the absence of over-specializztion to quantized prediction in the "Base" model preserves sensitivity perceptual distortions.
Table 5 narrows these models down to their top performing layer, chosen by the max-min criteria of the PCC and SRCC values, across all layers. A no-encoding option is also reported, where
21

---

Published as a conference paper at ICLR 2026
PS PCC across Layers
PS SRCC across Layers
100
100
90
90
80
80
70
70
PCC (%)
60
SRCC (%)
60
40
40
20
20
0
20
40
60
80
100
20
40
60
80
100
Transfomer layers (%)
Transformer layers (%)
PM PCC across Layers
PM : SRCC across Layers
100
100
PCC (%)
SRCC (%6.
70
70
60
60
20
40
60
80
100
20
40
60
80
100
Transformer layers (%)
Transformer layers (%)
wav2vec2 (large)
HuBERT (large)
WavLM (large)
wav2vec2 (base)
HuBERT (base)
WavLM (base)
Figure 6: For English mixtures, the effect of transformer layers in different pretrained self-supervised models on the PCC and SRCC values for the PS and PM measures.
waveforms are skip-connected directly into the diffusion maps, which under-performs compared to encoded modes and emphasizes the effectiveness of the waveform encoding in the proposed pipeline. These results reaffirm that shallow layers achieve optimal performance. Although point-by-poing comparisons show that "Base" models perform comparably to, occasionally slightly exceed "Large" models, applying the max-min criteria across the models in the table reveals that "Large" models are preferable when jointly optimizing for PS and PM. For once, wav2vec 2.0 Large achieves for the PM a PCC and SRCC differences from its "Base" counterpart of absolute 6% and 2%, respectively, even when the PS case shows negligible gap. Among the "Large" model variants, wav2vec 2.0 Large with transformer layer 2 emerges as the ideal configuration and we carry it forward as case study we investigate. It should be noted that among "Large" models. the PS very slightly changes with roughly 1% and 0.5% gaps between extremes for the PCC and SRCC, respectively, while the PM gaps are more meaningful. This suggests that the choice of model may mainly affect the PM scores.
We also investigate into how the SRCC and PCC deterministic and probabilistic error bounds behave under change of models, as shown Table Three main observations emerge:
(i) Deterministic errors are uniformly small across all models. The deterministic radii remain small for every model (less than 0.2% for SRCC and 0.21% for PCC under PS, and less than 0.26% for SRCC and 1.38% for PCC under PM). This sustain the observation made in the paper, which that the deterministic uncertainty around the PS and PM correlations small enough so that the competitive ranking of the measures is sustained and is not driven by unstable estimates.
(ii) Uncertainty is highly comparable across models. In the PS case, the CIs for SRCC vary only from 26.47-30.03% and for PCC from 9.10-10.299 across all models. Similarly, in the PM case, the CIs are a narrow band for most models (e.g.. SRCC 7.23- 1.33% for five of the six models, PCC 3.83-5.364, for four of the six). No family of models out of wav2vec2, hubert, and wavlm
22

---

Published as a conference paper at ICLR 2026
Measure
Representation
Transformer Layer
SRCC
PCC
PS
wav2vec 2.0 (Large)
84.12%
83.74%
PS
wav2vec 2.0 (Base)
84.25%
83.23%
PM
wav2vee 2.0 (Large)
2
84.69%
86.36%
PM
wav2vec 2.0 (Base)
2
82.79%
80.07%
PS
WavLM (Large)
3
84.80%
84.16%
PS
WavLM (Base)
84.84%
84.19%
WavLM (Large)
85.71% %
81.44%
PM
WavLM (Base)
2
82.82%
77.51% %
PS
HuBERT (Large)
3
84.48%
83.09%
PS
HuBERT (Base)
84.83%
82.73%
HuBERT (Large)
84.12%
82.24%
HuBERT (Base)
81.37%
79.47%
PS
Waveform (raw)
73.42%
71.04%
PM
Waveform (raw)
69.30%
66.62%
Table For English mixtures, comparing PCC and SRCC values between best-layer performance of "Large" and "Base" models. A raw waveform option, i.e. no encoding, is also reported. The highest SRCC and PCC are in bold per PS and PM.
Representation
PS Radius
PS CIs (95%)
PM Radius
PM CIs (95%)
wav2vec 2.0 (Large)
0.16 / 0.21%
30.03 10.29%
0.11 0.99%
7.23 3.83%
wav2vec 2.0 (Base)
0.16/ / 0616%
28.35 99.91%
0.01 1.10%
11.33 5.36%
HuBERT (Large)
0.07 / 19%
26.47 99.66%
0.15 / 1.38%
8,44 4.30%
HuBERT (Base)
0.20 / 10 16%
28.34 / 99.11%
0.07 1.24%
11.08 5.16%
WavLM (Large)
0.09 / 0.20%
26.94 / 9.411
0.26 / 1.24%
9,99 4,8 81%
WavLM (Base)
0.00/.11%
26.66 99.0%
0.25 1.38%
16.13 /.222%
Table 6: PS and PM radius values and corresponding 95% confidence intervals for different self- supervised representations (Large Base). Values reported SRCC/ PCC %.
systematically exhibits larger error bars, indicating that the different front-ende are estimated with essentially the same level of reliability.
(iii) Larger variants tend to slightly more stable than their base counterparts. Within each family of models, the "large" variants have consistently smaller PM CIs than the corresponding base model (e.g.. for Cls in the PM case wav2vec2 large gives 7.23/3.83% and wav2vec2 base gives 11.33/5.36% and similar patterns hold for hubert and wavlm). This suggests that increased model capacity improves stability rather than inflating uncertainty.
Next, we analyze all scenarios with wav2vec 2.0 "Large" encoders for speech and the MERT encoder for music representations, and analyze the effect of their transformer layer on performance. The results are shown in Figure 7 English and Spanish mixtures, both evaluated with wav2vee 2.0 backbones, show broadly similar trends across layers, with Spanish exhibiting a sharper decline in deeper layers. This can be explained by the XLSR pretraining data being relatively scarcer in Spanish than in English, leading later layers to emphasize cross-lingual abstractions over fine acoustic detail (Conneau et al.) 2021). Music mixtures with drums show the lowest performance among scenarios, which we attribute to the dominance of strong percussive transients. Self-supervised models have demonstrated less stability these highly non-statinn non-stationary regions, reducing the ability of PS and PM to capture perceptual degradations (Zeghidour et al., 2021). In contrast, music mixtures without drums demonstrate consistently high performance, in most layers even surpassing speech mixtures. This likely stems from the MERT backbone being particularly suited in capturing harmonic and timbral structure, allowing the measures to remain faithful to perceptual cues such as instrument texture and vocal clarity (Li et al. 2024). Interestingly, whether drums are present not, MERT-based performance demonstrates a steady behavior across all layers, suggesting the MERT representations not vulnerable to degradation across processing stages. The max-min min criteria across all layers, per scenario, shows that the ideal layers for English, Spanish, drums, and no-drums music mixtures layers 2, 2, 1, and 3, respectively.
23

---

Published as a conference paper at ICLR 2026
PS PCC across Layers
PS: : SRCC across Layers
100
100
90
90
80
80
70
60
60
PCC (%6
50
SRCC (%)
50
40
30
30
20
20
10
0
-10
-10
20
0
20
40
60
80
100
20
40
60
80
100
Transformer layers (%)
Transformer layers (%)
PM PCC across Layers
PM SRCC across Layers
100
100
90
90
O
80
80
PCC (%6
70
SRCC
60
60
50
50
40
30
30
0
20
40
60
80
100
20
40
60
80
100
Transformer layers (%)
Transformer layers (%)
English
Spanish
Music (Drums)
Music (No Drums)
Figure For all scenarios, the effect of transformer layers in their respective pretrained self- supervised architectures on the PCC and SRCC values for the PS and PM measures.
We employed these layers to construct Table I in the main text and now we extend the discussion on it. The advantage of PESQ can be attributed its long-stane standing perceptual model, which explicitly encodes aspects of loudness perception, asymmetry, and time-alignment penalties, features that directly penalize separation artifacts. In Spanish mixtures, the and PM are most performant in terms of PCC, but fall behind PESQ and SDR-based metrics in SRCC. One possible explanation that the syllable-timed rhythm and steady vowels of Spanish make fidelity-driven metrics such as SI-SDR, CI-SDR, and SDR more predictive of listener rankings, these metrics emphasize reconstruction accuracy at the waveform level. For music mixtures, PS and PM achieve the strongest overall correlations across both drums and no-drums conditions for both PCC and SRCC. Even though SpeechBERTscon has shown impressive results and is also based on a self-supervised backbone, is mostly competitive with measures, and notably even performing worse than raw waveform version at times, which projects on the importance of the diffusion maps in the pipeline. We emphasize that unlike English. we only inspected one backbone model for Spanish or music mixtures. In addition, the aggregation strategies we applied were not data-driven but heuristic and reasoning-basee choice. Consequently, while the proposed measures already demonstrate strong alignment with human perception, these low-hanging fruits may potentially boost performance. Quite surprisingly, the first group STOI, PESQ, and SDR-based measures is consistently preferable to the second group consisting of DNSMOS, BpeechBERTsscre. UTMOS, and others, which rarely achieve more than 70% in performance. One crucial conclusion this table suggests is that measures originally developed for certain audio application, should not be zero-shot adapted into other applications, and in that case into source separation evaluation. Otherwise, values that drift from human opinion may be reported, which may spiral the development of audio technologies instead of accelerating it.
24

---

Published as a conference paper at ICLR 2026
PS PCC across Misalignment
PS SRCC across Misalignment
100
100
90
PCC (%6
SRCC (%)
80
70
60
50
50
40
40
0
10
20
30
40
50
60
70
80
90
100
10
20
30
40
50
60
70
80
90
100
Misalignment (ms)
Misalignment (ms)
PM PCC across Misalignment
: SRCC across Misalignment
100
100
90
PCC (%)
80
SRCC (%)
80
70
70
60
60
50
50
40
10
20
30
60
70
80
90
100
10
20
30
40
90 100
Misalignment (ms)
Misalignment (ms)
English
Spanish
Music (Drums)
Music (No Drums)
Figure 8: The effect of temporal misalignment between references and outputs of the separation system on the PS and PM measures.
An additional stress test for our measures concerns their robustness to temporal misalignment between the input and output streams of the separator, a phenomenon commonly introduced by modern communication systems per-speaker microphones, such as meeting datasets (Carletta al., 2005 Vinnikov et al. 2024). Figure üigurrBillustrates illustrates the effect of artificial delays ranging from ( ms to 100 ms across English, Spanish, and music scenarios. While performance gradually degrades for speech scenarios as misalignment grows, as expected, 20 ms delay less still preserves coefficients higher than 80%. Surpassing this threshold, however, often causes pronounced drop that underscores this weakness in our measures, since human ratings insensitive these short latencies. Music mixtures exhibit different pattern, as performance remains largely stable across delays, with the presence of drums introducing more variability than its counterpart.
e.g., when dealing with references obtained from different,
Table 7 provides complementary information to the NMI results in Figure but listing the frame counts used for the PS and PM measures for every examined threshold. Even the lowest threshold of 0.1, had a minimum of 481 for calculations, rendering its results statistically reliable. An interesting observation in the music scenario, without drums, that it exhibits significantly more time frames in which there are at least two active sources, compared to all other scenarios.
In the next phase, we investigate the deterministic error radius and probabilistic Cls derived for the PS and PM measures in Appendix E Figure P shows histograms of the frame-level error distributions for speech and music mixtures. As expected, maps process is typically an order of magnitude smaller than the 95% probabilistic width, which is originated from finite-sample clusters on the manifold. The error radius also concentrated mostly near zero, which further confirms its negligibility. Cls typically span 10-50% of the dynamic range of the measures at the frame level, but surprisingly in the PM, between 10-15% frame-level instamces have probabilistic tails that approach zero across scenarios. The immediate contribution of these results are by making development of source separation systems more reliable and informed at the frame-
radius caused by the spectral truncation the diffusion
25

---

Published as a conference paper at ICLR 2026
Threshold
English
Spanish
Music (Drums)
Music (No Drums)
PS
PM
PS
PM
PS<th
PM<th
PS
PM<
0.1
583
7426
622
10721
481
13987
622
22859
0.2
1546
12086
1591
15756
1536
16126
1832
28828
0.3
3191
16350
3350
19714
3327
17846
4226
33231
0.4
5753
20118
6091
23054
5861
19492
8821
36953
0.5
9115
23697
9725
25964
8927
21033
15748
40426
0.6
13364
27232
13904
28627
12037
22522
24958
43769
0.7
18465
30758
18592
30885
15373
23864
35434
47186
24477
34076
23871
32703
19498
25168
46078
50682
0.9
31572
36507
29589
33902
25748
26614
57795
54939
1.0
37888
37888
34496
34496
34688
34688
66528
66528
Table 7: Frame counts used for NMI computation at each threshold, denoted 'th' in the table. Columns show counts of frames per scenario, split by PS and PM subsets.
PS: Deterministic Error Radius
PS: Probabilistic 95% Confidence Bound
50
40
Density (%)
Density (%)
30
20
10
0.01
0.02
0.03
0.04
0.2
0.4
0.6
0.8
Radius
Bound
PM: Deterministic Error Radius
PM: Probabilistic 95% Confidence Bound
30
15.0
25
12.5
Density (%)
Density (%)
20
10.0
15
7.5
10
5.0
5-
2.5
0
0.005
0.010
0.015
0.0
0.1
0.3
0.4
0.5
0.6
Radius
Bound
English
Spanish
Music (drums)
Music (no drums)
Figure 9: An histogram view of the frame-level deterministic error radius and the 95% probabilistic tail the PS and PM measures across scenarios.
For illustration, Figure shows reference and output spectrograms from an English mixture, time- aligned to corresponding PM and PS values over a 10-second utterance using the "Large" models, with layers specified in Table 5 While single example cannot be over-interpreted, the latest observation about the PS gaps across layers visually supported here, with very similar behavior of all models. The PM shows highly correlated behavior, but with noticeable different values by wav2vec 2.0, which exhibits the highest PM value for PCC. An interesting visual example is shown approximately the 9 seconds mark, when both speakers exhibit visible self-distortion artifacts
26

---

Published as a conference paper at ICLR 2026
Reference (Speaker 1)
Freq (H2)
5000
0
Reference (Speaker 2)
Free (f2)
5000
(Speaker
Freq (H2)
5000
Outout (Speaker 2)
Freq (Hz)
5000
0
PM (Speaker 1) 6h
PM
0.5
0.0
PS (Speaker 1)
1.0
ma yfth
0.5
0.0
(Speaker
1.0
M.
An lr
A1.
0.5
A
0.0
PS (Speaker 2)
1.0
/ mm
A
M. NM
0.5
0.0
10
Time (s)
wav2vec2
HUBERT
WavLM
Figure an English mixture with two speakers, spectral view the system signals and aligned with time-series view of the PS and PM measures of each speaker across different self-supervised architectures. Blank time intervals remain whenever speech does not overlap.
accompanied by sharp drops in their PM measures. Listening tests confirmed that leakage is indeed more present in "Speaker 2" than in "Speaker 1", supported by the PS plot.
27

---

Published as a conference paper at ICLR 2026
Finally, to give the reader an intuitive grasp of how the two error terms evolve in time-aligned manner with the PS and PM measures, Figure 11 fillustrates a representative example.
1.0
0.6
0.2
1.0
0.6
0.2
0.010
0.005
0.000
0.008
0.006
0.004
0.002
ar
0.000
PM probability
probability
0.4
0.2
2
6
Time
Speaker
Speaker 2
Figure 11: Time-aligned view of the PM and PS measures and their deterministic error radius and probabilistic tail with 95%, two English speakers. Time indices where speech does not overlap remain blank.
28

---

Published as a conference paper at ICLR 2026
PM vs. PESQ Temporal Misalignment (ms)
Lang.
Metric
0
10
20
30
40
50
60
70
80
90
100
EN
PM
1.00
0.91
0.84
0.81
0.80
0.79
0.79
0.79
0.79
0.79
0.78
EN
PESQ
4.64
4.49
4.43
4.40
4.41
4.36
4.35
4.33
4.33
4.32
4.29
SP
PM
1.00
0,93
0.89
0.87
0.86
0.86
0.86
0.86
0.86
0.86
0.86
SP
PESQ
4.64
4.59
4.50
4.47
4.46
4.39
4.37
4.33
4.31
4.27
4.25
PS vs. PESQ
Packet Loss (%)
Lang.
Metric
0
2.2
4.4
6.6
8.8
11.11
13.33
15.5
17.8
20
EN
PS
1.00
0.93
0.94
0.93
0.93
0.93
0.97
0.96
0.95
0.94
EN
PESQ
4.64
2.45
1.39
1.17
1.14
1.08
1.07
1.05
1.04
1.04
SP
PS
1.00
0.93
0.94
0.93
0.93
0.93
0.97
0.96
0.95
0.94
SP
PESQ
4.64
2.91
1.53
1.22
1.17
1.08
1.05
1.04
1.04
1.04
Table 8: PM and PS behavior compared to PESQ under temporal misalignment and packet loss, for English (EN) and Spanish (ES) trials.
We now compare PS and PM to PESQ - our strongest comparator as of Table I PS and PM operate on diffusion- map distances computed from self-supervised embeddings, using bank of synthetic distortions around each reference. They are explicitly designed to (i) disentangle leakage from self- distortion, and (ii) function at the frame level. PESQ, in contrast, produces a single utterance-level MOS-aligned score using engineered auditory features and an internal time-alignment stage (Rix et al. 2001). As shown in misalignment stress-test in Fig even moderate delays (20 ms or more) between reference and output reduce PM because it compares frame-synce came--synchooous embeddings without an alignment stage. PESQ, in contrast, performs time alignment and is largely insensitive to such shifts. Thus, a separator that outputs a clean signal with 30-100 ms latency, common in practice and imperceptible to listeners, receives high MOS and high PESQ, but PM drops because each frame is matched to the wrong reference frame, inflating Mahalanobis distances. Our experiment models this exactly: pure time-shifted reference with no added leakage or distortion, consistent with real- real-time communication standards that allow up to 40 ms latency. PESQ is explicitly built for telephony/VolP and includes mechanisms for packet loss, time alignment, and "bad-frame" handling. PS, by design, targets continuous distortions and does not model deletions, insertions, non-uniform time warping. In our test, the system output generated by randomly deleting 20 ms frames from the reference and then time-compressing and resampling to the original length, without any leakage or distortion.
In Table 8. we report average PM and PESQ values for delays from 0-100 ms, and average PS and PESQ values for packet-loss rates from 0-20%, averaged across trials in each SEBASS speech scenario (English and Spanish). Under misalignment, PESQ decreases modestly from 4.64 to 4.29 (English) and 4.25 (Spanish) at 100 ms-only 5% of its dynamic range. P.M. bounded in [0, 1]. degrades far more: 22% (English) and 14% (Spanish), showing substantially higher sensitivity to timing shifts, whereas PESQ remains closer to human judgments. For packet loss, PESQ collapses by more than half its dynamic range even below 5% loss and approaches I 20% loss, aligning with the severe perceptual degradation reported by listeners. PS, by contrast, changes only modestly, indicating that it does not fully capture the perceptual impact of packet loss.
We now ask: How well does MAPSS generalize when one distortion group is entirely left out, and another is augmented with an excessively strong out of distribution variant? We therefore conducted the following experiment. MAPSS uses 13 distortion groups, each with several parameterized variants about 70 total distortions per reference. For each of the 13 groups, we repeat the same two-step procedure: (i) remove the entire group from the distortion bank, and (ii) augment a different remaining group with one new, deliberately excessive parameter whose effect is perceptually far from the reference. For example, we omit the "tremolo" group and introduce additional high-severity noise parameter (e.g., additive noise at -20 dB SNR) into the "noise" group. In Table 9 each row specifies which group ("E."). As in the main paper, each entry reports the SRCC/PCC values (in %) obtained by averaging the aggregated PS and PM scores across all trials for that scenario.
omitted ("O.") and which group received the new excessive parameter
29

---

Published as a conference paper at ICLR 2026
Scenario
PM (SRCC/ PCC %)
English
Spanish
Music w/ Drums
Music w/o Drums
Original
84.69/86.36
83.41/85.30
75.18/69.88
88.12/85.26
O. notch, comb
82.13/81.8 1.62
79.29/80.16
74.57/69.83
88.59/85.26
O. comb, E. tremolo
81.75/81.11
79.09/79.61
74.89/69.99 92
88.47/84.98
O. tremolo, noise
81.57/80.85
78.82/78.8
74.34/68.33 34
88.27/84.66 4.60
O. noise, E. harmonic
82.20/77.83
80.14/76.66
75.06/67 40
90,48/80.65
O. harmonic, reverb
81.72/81.27
79.08/79.75
75.16/70.05
88.81/85.00
O. reverb, E. noisegate
82.18/81.74
79.45/80.32
75.06/69 888
88.50/85.22 2
O. noisegate, pitch
82.13/81.6 .64
79.13/79.69
74.97/69.77
88.48/85.00 06
O. pitch, E. lowpass
81.66/81.20
79.33/79.38
75.05/70.06
88.44/84.70
O. lowpass, E. highpass
81.46/80.54
78.06/78.73
75.00/69.79
88.82/85.19
O. highpass, E. echo
81.44/81.02
79.05/78.45
76.22/70.05
88.59/85.00
O. echo, E. clipping
82.19/81.74
79.47/80.31
75.47/69.94
88.40/85.26
O. clipping, E. vibrato
82.04/81.53
79.32/80.00
75.14/69.85
88.47/85.13
O. vibrato, E. notch
81.84/81.43
79.34/79.86
75.29/69.89
88.54/85.02
Scenario
PS (SRCC PCC %)
English
Spanish
Music w/ Drums
Music w/o Drums
Original
84.12/83.74
82.33/85.01
72.87/77.38
87.23/87.8 881
O. notch, E. comb
84.09/83.52
82.67/85.08
70.65/76.38
85.56/86.73
O. comb, tremolo
84.27/83.52
82.45/85.09
70.86/76.62
86.02/86.99
O. tremolo. E. noise
84.30/83.41
82.52/84.97
68.96/75.68
85,81/86.5
O. noise, E. harmonic
84.51/83.95
83.49/85.08
72.12/77.41
87.17/87.97 97797
O. harmonic, E. reverb
84.23/83.56
82.60/85.13
69.80/76.60
85.89/87.00
O. reverb, E. noisegate
83.95/83.34
82.51/84.98
70.20/76.43
85.68/86.69
O. noisegate, pitch
84.38/84.43
82.81/85.74
70.54/76.63
86.05/86.82
O. pitch, E. lowpass
82.64/82.32
81.22/83.70
70.87/76.61
85.52/86.98
O. lowpass, E. highpass
84.34/83.55
82.85/85.04
70.76/76.56
85.48/86.77
O. highpass, E. echo
84.58/83.49
82.52/85.21
71.39/76.24
86.01/87.01
O. echo, clipping
84.20/83.46
82.33/85.02
70.50/76.40
85.70/86.68
O. clipping, E. vibrato
84.32/83.50
82.43/85.04
70.66/76.50
85.53/86.81
O. vibrato, notch
84.11/83.48
82.60/85.05
70.16/76.50
85.91/86.77
Table 9: PM and PS performance (SRCC/PCC) across all distortion pairs and content types.
This experiment directly probes the generalization of MAPSS when the distortion bank itself made strongly out-of-distribution with respect to the artifacts produced by the separation systems. For each of the 13 distortion groups in the paper, we (i) remove that entire family from the bank, so MAPSS never "sees" this type of distortion when constructing its manifold. and (ii add an excessively strong parameter to a different family. The SEBASS mixtures, separator outputs, and human scores remain fixed and only the perceptual model used by PS and PM is perturbed. Deviations in SRCC/PCC therefore quantify how much MAPSS relies on a correctly specified distortion bank versus how well extrapolates unseen mis-calibrated distortions. Across all 13 omit/extend configurations and four SEBASS scenarios, PS is well-generalized. Relative to the original configuration, PS SRCC/PCC vary by at most 1.5/1.4 points for English and Spanish speech, 3.9/1.7 points for music with drums, and 2/1 points for music without drums. In all cases, PS keeps the same qualitative ranking against competing metrics as in Table of the paper: remains among the best-performing measures in terms of correlation with human mean opinion score. This indicates that the separation score generalizes well even when the manifold is built from a substantially misspecified distortion bank and when one distortion family is pushed far beyond the severity range used in the original setup. PM shows a clearer limit to out of distribution generalization, which expected because it explicitly models the empirical distribution of self-distortions within each cluster. When remove perceptually dominant family for speech, most notably additive noise, and simultancously inject a very severe parameter into another family, the largest degradation is about 3-5 SRCC points and 8-9 PCC points
30

---

Published as a conference paper at ICLR 2026
for English and Spanish. For other omit/extend pairs the effect is smaller (typically 1-4 points), and for music PM is essentially stable: with drums, SRCC changes by less than point and PCC by 2.5 points, and without drums, SRCC never decreases relative to the original configuration and only one condition reduces PCC by 4.6 points. These results suggest that PM extrapolates gracefully to much stronger distortions within the families represented in the bank, but its performance degrades when the bank completely omits a distortion family that strongly influences human ratings in that domain (e.g., noise in speech). We argue that the main practical limit we observe is coverage of perceptually salient distortion types, not sensitivity to the exact parameter ranges.
English
Spanish
Music no drums
Music drums
0.96/1.00 .00
0.96/1.00 1.0
0.96/1.00
0.94/1 ,00
0.11
0.91/0.91
0.88/0.90
0.70/0.83
0.85/0.90
0.22
0.83/0.88
0.79/0.87
0.61/0.83
0.76/0.87
0.33
0.76/0.87
0.72/0.85
0.55/0.83
0.70/0.85
0.44
0.69/0.85
0.65/0.84
0.50/0.83
0.65/0.85
0.55
0.64/0.85
0.59/0.84
0.45/0.83
0.61/0.84
0.66
0.60/0.84
0.54/0.83
0.42/0.83
0.58/0.84
0.77
0.56/0.84
0.50/0.83
0.39/0.83
0.55/0.83
0.88
0.53/0.84
0.47/0.83
0.36/0.83
0.53/0.83
0.50/0.83
0.44/0.83
0.34/0.83
0.51/0.83
English
Spanish
Music no drums
Music drums
1.00/0.99
1.00/1
1.00/1.00
1.00/0.98
0.11
1,00/0.94
1,00/0.95
1.00/0.98
1.00/0.91
0.22
1.00/0.90
1.00/0.90
1.00/0.96
1.00/0.86
0.33
1.00/0.85
1.00/0.84
1.00/0.94
1.00/0.79
0.44
1.00/0.79
1.00/0.78
1.00/0.91
1.00/0.72
0.55
1.00/0.73
1.00/0.71
1.00/0.86
1.00/0.63
0.66
1,00/0.66
1.00/0.64
1.00/0.79
1.00/0.54
0.77
0.99/0.58
0.99/0.56
0.99/0.69
0.99/0.42
0.88
0.99/0.48
0.99/0.46
0.99/0.56
0.99/0.30
0.98/0.36
0.98/0.34
0.98/0.40
0.98/0.17
Table 10: Sensitivity analysis over and À parameters (SRCC/PCC in %).
By design, PM only measures proximity the target reference and its distortion cloud, while PS also compares to non-attributed references. To show that this construction disentangles leakage from self-distortion, include two new controlled experiments where leakage and self-distortion vary independently. For each mixture 2 = //1 + 1/2 we construct outputs Φ1 (ax) y1 + az/2 for a E [0, 1]. Here only leakage increases with In second experiment, select new distortion not present in the MAPSS distortion bank and apply to J/1 with continuously increasing strength À E [0. 1). Here only self-distortion increases with A. Table provides a compact summary table. Each cell the aggregated PS/PM, averaged across trials in each scenario, over the corresponding or À values. We observe that PS responds selectively leakage and PM responds selectively to olf-distortion, with minor false reaction to leakage.
English Only
Spanish Only
Multilingual
PS (SRCC/A PCC %)
84.12/83.74
82.33/85.01
86.27/85.60
English Only
Spanish Only
Multilingual
PS (SRCC/ PCC %)
84.69/86.36
83.41/85.30
84.41/84.07
Table 11: Comparison between monolingual and multilingual construction the MAPSS pipeline for PS and PM (SRCC/PCC in %).
31

---

Published as a conference paper at ICLR 2026
Real-world scenarios are often multilingual and that purely monolingual experiments limit MAPSS's practical relevance. To address this, we evaluated MAPSS on multilingual mixtures from SEBASS: original 4-speaker mixtures in two configurations, (i) four female speakers (two English, two Spanish) and (ii) four male speakers (two English, two Spanish). Complementing the monolingual results of T we now report TT] PS slightly improves in the multilingual case, likely because clusters are formed over all sources, yielding a richer manifold on which to assess fargetinnteffrence relations (e.g., fine-grained confusions between voices and languages). PM remains roughly unchanged. depends only on the target cluster and not on cross-cluster geometry. Together, these results suggest that with respect the SEBASS database, MAPSS is least as stable, and in PS's case, slightly more informative, in multilingual mixtures than in monolingual ones.
D EXPECTATION AND PROBABILISTIC CONFIDENCE BOUND OF THE TRUNCATION ERROR
Truncating the spectrum to d dimensions breaks the equality in Equation P and leads truncation error. Here, we derive the expectation and probabilistic tail bound for this truncation error. Assume point x; E X is drawn from the stationary distribution 7 of the diffusion process, where i E ( J.. This assumption is supported by (Hein et al. 2005 Lem. 1), and by showing empirically on 5, 000 graphs that the corresponding eigenvector matches the theoretical stationary distribution up to statistical fluctuations. Given that the N 1-dimensional embedding of x; is truncated dimension d, then the truncation error is expressed as (10)
1/2
E(x,) =
Σ
X2'
uZ)
(62)
(=d+1
We define the squared truncation error and analyze it:
N-1
T(x) (x.)
Σ E=d+1
X2 (t).
(63)
Since the eigenvectors N-1 are orthonormal under π,, then:
Er (i) Σ (z) = 1,
(64)
from which we derive the expectation of Txx,) under n:
N-1
Ex [T(x)] = FT
Σ
x2
Σ
(65)
2=d+1
tdd+1
Thus, the expectation of the truncation error is given directly by:
N
1/2
Ex [E(x)]]
=
2
(66)
This term decays monotonically as d grows and is typically lower than 10-3 To obtain a non- asymptotic and nigh-probabiiity confidence bound on the truncation error, derive VE, i (64)
-1/2 [ S Fmin
Tiin
min (E()....N)
(67)
Any bounded variable is sub- Gaussaas,,n and its norm is most the bound divided by Vn2 /Vershynin 2024 Example 2.6.5):
Amin 1/2
(ue(i) 1122.
== K.
(68)
Vln 2
Let m N 1- d, so we define z, E Rm as:
2, (1)+1 ..., un uN-1 u (,
(69)
32

---

Published as a conference paper at ICLR 2026
and the diagonal matrix of weights D I mmmx
D diag( N d-1 d....... -1--)).
(70)
Then Exx)) and Txx) can be rewritten as:
T(x) E(x;) = Dz;
Dz
(71)
2'
(72)
For L > 0, ue( is zero-mean under Consequently, the vector is zero-mean and by definition satisfies ||zi|l02.0 I K m.. We also notice that multiplication by fixed matrix scales the sub- Gaussian norm linearly, and since D is symmetric and positive:
[Dz(i)ll K [K/miDD|
K Kym X K Kvma444 E>d
(73)
According to Vershynin 2024 Prop. 6.2.1), for an m-dimensional, zero-mean and Sau-Gaussian vector Y with
1 k, it holds:
P (I 2 Cx((m+++ 1)} S Se-
(74)
where 2 0 and C >> 0 is a constant. Setting Y = Dz; and k. K Kymd/44 gives:
P P T(xs) > C2 K2 K2 Vm + +))²} S se-
(75)
Let 5 E (0, 1) and set vln(1/8). We rewrite (75
2 (Vm+ v
P T(x)) C2 Kmm
2 1 5.
(76)
Thus, the desired confidence bound on the truncation error is:
1 } F. E(x,) & CXa K m (m+ m/mln
21 1-8.
(77)
The choice of dimensions affects both m that shrinks linearly with and X4 that falls monotoni- cally with d. K is affected by the minimal stationary probability miin- so if the graph contains rare points then miin may be tiny, while well-balanced graph derives K VN and tightens the bound.
E DETERMINISTIC ERROR RADIUS AND PROBABILISTIC TAIL BOUND OF THE MEASURES
We derive deterministic error radius and a high-probability confidence bound on the frame-level PS and PM measures by considering: (i) spectral truncation error due to retaining d diffusion coordinates. which is separately developed in Appendix D (ii) finite-sample uncertainty in estimating the cluster centroid and covariance. We then combine these via union bounds. In this section, we consider fixed trail I, separation system g, and time frame f.
E. THE PS MEASURE
Considering source indices j E (1....,N, ($2). we begin by analyzing the effect of the truncation error, assuming access to cluster statistics. The difference between the embedding of X; and the centroid of cluster can be expressed in the truncated subspace R& and in its complement subspace
RRm respectively denoted (4) and A (m) Using 10 15
t.j
V
(Xx)
(d)
Rd
(78)
(m)
v
(m)
(x:) p
(m)
E 1
(79)
33

---

Published as a conference paper at ICLR 2026
where = N - d 1. For completion, for every X' and its global index & E {1, f1....., NJ:
(m)
(80)
c)fm)
mm) = 1 (mTL)
(xz))
L (m) (xj:n)
p= 1,.... 1.....AAp
Np}
(81)
(m) (x) = (A u (K)) 1
(82)
In the full, N 1-dimensional space, the cluster Cj is given by:
C (X)). (x(x))))
1, Nph.
(83)
with mean μ E RN-, difference A j.j
RV- and covariance Σj (N that hold:
mj (d) (m)
Σ (d)
Cj
Σ
(84)
(m)
CT
Σ (m)
where I)m) E Rmxm and Cj E Rd are:
Σ (m)
1
Σ
(v (1j
14 (m)) (vb (mm) )
(85)
N
1
(x,,n) (xx,)) - /1) (xx,))) -1 ) T
Cj
(86)
mm)
p=0
According to 16 the squared Mahalanobis distance from (X;) to Cj is:
Las (4,, T Σ + =z((N-1) -1 Aij.
=
(87)
where inversion is empirically obtained by taking 10-6 with (MV being the N - 1-dimension identity matrix. To evaluate the truncation effect, we perform blockwise inversion on (87) via the Schur complement (Horn& & Johnson 2012):
Lss ( m,) =
> (c) + E
1 + Sr 'res
(88)
where Ti, E RT and the Schur complement Sj E Rm hold:
Túj
(m) i
CT7
[ ( E()( -1 (d)
(89)
S, ) (m)
-C (2)0) + (l
Cj.
(90)
We now utilize the inequality:
Ya, b.b20 0: Va va S vb,
(91)
obtained by the mean-value theorem for f(-) = v- Rudin 1976 Ch. 5). Let us set:
(A T ( + = -1 (d)
A
(92)
ri Piis
(93)
to obtain:
|ōاف = |dw (x:); pej Eg) - dm (x)): ( S
VTT ST Tag
(94)
Namely, | is the truncation error of this Mahalanobis distance. From 17 it holds that:
83. A; A(4).
8..3- B, B34))
(95)
34

---

Published as a conference paper at ICLR 2026
where j is defined in 17 and:
A; = das (V.(x)) p4i, Σ,),
(96)
B; du (V,, (R:); Man 2g-))
(97)
Consider the N 1-dimensional representation of PS. (d) i.e. PS; (18)
A PS; = A, + B;
(98)
which is smooth and differentiable in A, and B,, since by definition A, + B; >> 0. We assume and have empirically validated that truncation introduces a small relative change, i...:
|o,,e < A!d0) B(
(99)
|0.55 < A (d) + B!a
(100)
(A! ((()) valid. We can therefore write:
making the first-order Taylor expansion of PS; around
PS; -Ps( ~
ÖPS;
(A( BB!d)
80,5 +
YPS;
(A!4) _.!.) 8es
(101)
0A,
0B,
1,00)
Oi +
(4/0) B!a)
(A( + B
where the quadratic remainder in the expansion is empirically one order smaller than the first-order
term and can be safely dropped. Applying the triangle inequality and (4 yields the deterministic error radius in the PS measure:
B.d
[6,, + A!40 18
B!a
T V TT..S. ri A! V r.. Sj.t
PS: - PS S
(102)
11 ((())
(1(0) + B(a))
We now quantify the uncertainty in PS; due finite-sample cluster statistics. Empirically, observe that cluster coordinates exhibit weak dependence between one another and derive from (Bartlett, 1946) the following cut-off rule for the effective sample size of cluster cl
n
n2 = (
j.elfl
(103)
1 2
Σ Pz.e
where PJ.e is the empirical average Pearson auto nuto-correlation of coordinates lag e, and:
L5 arg min
lpg.cl
20.975
(104)
n2
Empirical evidence across 5, 000 graphs suggest that on average E, Pi.e = 0.2, and so we set off 0.7 n, for all clusters.
To bound the deviation between the estimated and true cluster mean and covariance, we employ the vector and matrix Bernstein (Vershynin [2024] Props. 2, and the dependent Hanson-
Wright inequalities (Adamczak 2014 Thm. 2.5). For every ". 7. (0, with respective
least probabilities -8,5 and
2dmax (dd)) In (2/8PS
j.pa
[1y - -p,1
= A
(105)
eff
T3 + In 2/85S 3.E)
(0(00) -5/00 < C. (S(a))
Ti
(106)
Njjeff
Nj,eff
35

---

Published as a conference paper at ICLR 2026
with absolute constant C' 0 and the ratio:
tr (((0)
(107)
Amax )
Let us integrate the definitions of A! and B(
17 with 105
106p.
Then, with probability of at
least 1 P 0 A((0) and ( deviate from their true versions by rs bounded by:
(A and ESS (B(d))
(dd)
P8 ((d)
< 22
A
DiP
max
2
+
A(d)
A
(108)
X
Amin
()(0)
Amax
(((()
{i(d)) j'
PS
(
S 2
B Dr
Amax
+
(())
D
(109)
Amin
(())
Amax
((()
We avoid extremely loose bounds by replacing tiny, rarely observable eigenvalues, by a robust floor
eigenvalue. Given matrix A Redxd define Amin (A) (Horn & Johnson, 2012 Thm. 4.3.1):
Amio (A) Ammin (A + Am-maxxx (A) (())
(110)
where = 0.05 is typically taken and ((d) is the identity matrix. Ultimately, let us define the Euclidean Lipschitz constant Lip as:
(1 (B(.))
ÖPS; (A (A _BO)))
OPS; (A) B!!))) ().
L PS
(111)
aB,
(7(4) Bi(00)
which enables us to bound the finite-sample deviation of PS; (d)
with:
PS
PS(()
L
V
[7
+ eP (((a)
(112)
Finally, we employ the triangle inequality on both error sources (102 and 112 and obtain for
JPS
PPS x
8pS
.. with 5P E (0, 1):
P
PS
PS,
<
(113)
((d)
TT,S, rii + A(40)
Vi.j" S
pis"
L JPS
(7()) + ePS (B!(0)
2 1 8.5
(A(0) + B(())
We now analyze the obtained expression separately for the deterministic and probabilistic terms. In the former term, the two square-root terms energies that leak into the truncated complement after regressing out the retained d diffusion coordinates. Intuitively, {dd) encodes the local anisotropy of cluster j the kept coordinates, C, represents coupling of residual energy in the truncated block,
and Σ (m)) is the spread that remains in the truncated block. Therefore, larger S, down-weights complement deviations, reducing the bias, while ri.j and S, co-vary through the cross-covariance C,. Namely, increasing C, shrinks both тı,j and Sj. while decreasing C, does the opposite. The practical rule is prevent tiny Amia (Sy), e.g., by promoting such directions into the kept set via local choice of and shape distortions so the complement is predictable from the kept coordinates, keeping Fi.j small.
36

---

Published as a conference paper at ICLR 2026
For the probabilistic part, its width reflects uncertainty in the empirical centroid and covariance of the attributed and nearest foreign clusters, where Amax {max(((())) and nj,eff determine the width primarily. Practically, correlated distortions shrink Rfjeff and widen the bound, and smaller spectral flatness ratio Tj yields tighter matrix concentration. As expected, the probabilistic piece dominates the deterministic as shown in Figure 9 which underlines the importance of cluster construction and dependence control.
E.2 THE PM MEASURE
As in the PS case, start with the truncation error and assume access cluster statistics. Let us reconsider (78)-190), but with two adjustments. First, the cluster coordinates are centered around the cluster reference embedding and not the cluster mean. Given m N
define:
1 (xi.p) 1 (d) (xx) ER"
(114)
v (m) (x.p) L (x, E R". Second, the cluster now absent the reference embedding. Namely, the full N cluster is (33
(m7)
(115)
1-dimensional
č - \ (x()))
(116)
The cluster C. has difference A E N- for every p (1...., N and covariance Σ, E ((N-1)x(N-1) that hold (19)
(d)
S
(a)
A.p
Cp (m)
ΣS =
b
(117)
(m)
with {(m) E Rmxm and C E Raxm being:
{(m)_
(w v (()) (xx))
(xb (
m/m)))p))) (xx)
(118)
xkm))
N p=1
C = [(m]
(xi
(1( (x:)) (m) (xx.p) (xm.)())))))) - w/(m)(xx)
(119)
In N dimensions, the squared Mahalanobis distance from L (xx,p) to C, is given by 16
dhi L (x:,)) (, (x,). Σ. A
A S + CN
ipp
(120)
where as (87). inversion has been empirically obtained with c = 10-6 and the N 1-dimensional identity matrix I),-1), We again turn to the Schur complement (Horn Johnson, 2012) and
decompose (20):
dhi 1 (X..p): (x,)) Σ.)
(4
(d)
(1(0) + ))(
(d) + 5, Tip.
(121)
hp
with Tip E R. and the Schur complement S, E mmxx being:
Ti.p A (m)
_
5 (d)
+ +c((d)
(d)
(122)
i.p
S. ( :
{ + el( -1
C
(123)
Let us define the of squared Mahalanobis distances of cluster č, in dimension N - 1 as:
G: = { dir (xi,))));; (x:)), Σ.)
p p= p=
(124)
in accordance the truncated version of (. in 20). By employing (21)) for every p E (1,..... Np). we can bound the truncation error of the squared Mahalanobis distance as follows:
dir (X.,): (x:)), Σ.:) dhr (w. (d) (xi,);; v (xx), E!
(125)
T S Tig 5g.
S
37

---

Published as a conference paper at ICLR 2026
and the difference between the mean of the elements in G3 and 91a0) can be expressed as:
N No
Ng. - 10 G
Σ
Pip : Tip
(126)
g (d)
gEC:
nEG
p=1
N₂ 6g.p ōg.,
Similarly, we can express the deviation of the variance:
9 Mogte) )
a G
Σ
(g (0-Mo,) Ms.)
(127)
G
g(aa)
SEC;
geo
and with 126 and the Sauchy-Schwarce inequality, we can obtain:
Np
a
S
(20gur
(gg. + t
(S
(128)
9
N2
1
where Sanu = max, óg..p The Gamma-matching parameters in the truncated and full dimensions are
42
k((d)
g(())
h;
(129)
o g(4))
2
0 ())
0:
(130)
Jgg.
and their deviations can be bounded by considering
126
(128):
h
(())
S C1
gmax
Np
Ngg
(
8g.
(131)
Np
0 +
o2 g(
0, -(4) <
G.PP
Xmax
N
8G,0: 8:
(132)
with universal constants C1 C2 > Let the squared Mahalanobis distance from the output embed-
ding to the cluster be:
dss (w (X:); (x:), (1(1) = aj.
(133)
and employing 125) for the output embedding yields:
(x:): (x,). Σ
di
(d) ( (d)) (x,), 5:00 = -r ri.a := óg.a- (134)
As in 23 the PM definition in dimension N can expressed using the regularized upper incomplete gamma function (k;, = [kk, . (k:):
PM, = Q kiz 0
(135)
Consider the cuncation-inducce ellipsoid:
B: = (K', 8,, aq) : K - 1!(a)) S kg 00 - 0!0)
s 80, |o;
a S 55 }.
(136)
For 0k) (k, a/0). the gradient with the partial derivatives with respect to k. 0 and is:
1 k-
In (k)) 2(k,x))
T(k)
a x k-1e-z
OF(K)0,a)
(137)
0²
r(k)
1 I --1e-
0 r(k)
38

---

Published as a conference paper at ICLR 2026
where = a and (i()) is the digamma function. Since VF(K, 0, is continuous and bounded on the compact set B,, we set:
LB.
sup (VF(k) oc. (K.9,))EE.
(138)
To yield the bound on the PM measure due to truncation, we notice that both (k; pz. a.) and
( 00 aa!") lie in B,, and apply the multivariate mean-value theorem to yield the following:
PM, - !M! PM
< LR, (6 + 60 + +0g 09.00
1/2
(139)
(33 is monotonically increasing in k: and decreasing v, for k; x > 137). We assume that on B;, and for all (0, a) E B,, FF/Ok does not change signs, or otherwise we fallback to (139). Consequently, the maximal change of (k,,
However, this bound can be tightened. We notice that (kk; x:)
inside B, is attained at one of its eight corners, and (139) can be tightened to this PM error radius:
PM: - PM
Q ae/o)))00)) Q (2( (00) 10,0))
max (ko.
(140)
As in the PS case, we now analyze how the finite number of coordinates in cluster leads uncertainty in the PM evaluation. Let Rg be the maximal squared Mahalanobis distance in G., namely:
max g. JEG;
(141)
Again, similarly to the PS case, we utilize the vector and matrix Bernstein (Vershynin] 2024 Props. 2.8. 1, 1.7.1.1) and the dependent Sanshn-Wright inequalities (Adamczak) 2014, Thm. 2.5). Let us consider the confidence parameters b,p 8PM, 6 E 1/3), so with respective least proba- bilities of 1 ( 1
8PM 1 8PM.
In (2/8PM) 202 glu) 6,14
3R, In (2/8PM)
Pat/:: - Mot00/ S
Aijo.
(142)
Np
Np
215 In (2/6PM)
3R In (2/0TM)
1 ( - -00,00 s
= Ai.o:
(143)
No
N
In (2/8PM)
a; à s Rij
(144)
Np
Recalling the definition of (d) we empirically validate that Ar, Pogas expansions to k,(0) gld) around R((d) g(0)
((d)) from 129).
130
since by design Pa(an (ي(0) > 0, and since
giee, we can apply the first-order Taylor
respectively. Apply the triangle inequality to it gives:
(x(4)
2/1
1.(0)) [((d)
g(00
<
Ai,u
Aio
Ai,
+
Aio = Aik, (145)
do
Ø2
gli0)
g
00(0) - -5,00
<
()((0)
Ai.u
+
a0(())
Ai,
02
g
Ai
g
2ō
A
A
(146)
0p
Oo
0,00)
(
Empirically, rarely A,,s, A or Aia become extremely loose. To avoid this behavior, we practically regularize the box by setting:
AiA + min
(A,,,, 0.5k((1)
(147)
Age + min
Ai,,, 0.50
(148)
A a,a min
A (.5a(
(149)
39

---

In the probabilistic part, the local box Bloc aggregates two finite-sample pieces. The first is the uncertainty of the moment, with D and Ai, scale I/N, but are amplified by the maximal radius of Mahalanobis within the cluster RG, Heavy outliers increase RG and widen both bounds. The Published as a conference paper at ICLR 2026
Let us consider the local box of values:
B.uc { 0%, R/ k/(0) + Ai.e 0f [o(0) + 2..0 a
a (d) + i. }
(150)
As discussed earlier, Q(ki x)) is monotonically increasing in k and decreasing in for k; >> 0 137 Consequently, the maximal change of Q(k, x) inside 0 is attained at one of its eight corners. Thus, the finite-saple enror of the PM measure in dimension d is bounded by:
PM - PM; (d)
(ke. 0c/ Q (R.0) a/
a ///00//00))) )
max (No, (Ke,
(151)
Ultimately, we combine the deterministic error radius with the probabilistic width. 0
Let
PP PPM
MPd + 8PM. which yields for 6Pm E (0, 1):
PM (
P
P.M -
(152)
Q (lies ac/Oc) Q (a 10(00) |
max (2)),
Q (kies ac (c) Q (R a (0 |
2 6PM
max (ke.0e,aa)E
In the deterministic term, large cross-block coupling C. or residual spread (m) again directly inflate the error radius via the Schur complement.
second is the uncertainty of the distance of the output, contributed by Ái,a which is also proportional to R3 but scales by 1/ \/ Again, this emphasizes the importance of the design of distortions.
F ERROR RADIUS AND PROBABILISTIC CONFIDENCE BOUND OF THE PCC AND SRCC
In this Appendix, we propagate the frame-level error radius and probabilistic widths developed in Appendix E. and E.2 to the reported PCC and SRCC values.
We start by fixing a trial I, source separation system and time frame f. Let the indices of the active sources in frame be S and consider a source E st. The observation of measure P E (PS, PM), denoted k.. be decomposed as: f.f
o + JAL.
g.P.P
P. Sif
(153)
where:
J.J.P. = Pif
L..PP 1.f
q.P.P +14
+
(154)
and gq..P. is an unknown deterministic bias with a provided radius g.d.P if
such that:
12..P s,f
(155)
J...P with if given by either (102) or 140
Regarding the probabilistic side, we define:
9.l.P i,f
g.I.P P
(156)
where:
of
g..
L.P.P
P.P off
(157)
and Ex (gd.P))
q.P.P Thus, the two-sided probabilistic half-width p...
> can be interpreted as:
Pr
q...P
14.
9.P.P
<
g.lp Pi.f
2 1 - 5p = c.
(158)
40

---

Published as a conference paper at ICLR 2026
with 5 and the probabilistic bounds defined in (122) and (511) We abbreviate c from
Consider Zz- the normal quantile at level c* = (1 2), so we calibrate the half-widths scale to be:
a.P.P
(159)
with tails still reported back as half-widths at the original confidence
We now propagate these errors from frame to utterance level, based on the aggregations we introduced in (45) and (49). average, experiments showed that frames more than = apart are effectively independent both for speech and music mixtures. Given the set F of time frames with two or more active sources, the standard Bartlett block-decimation (Bartlett 1946) yields the conservative inflation:
1/2
Ft F C
Fi Σ f=1
(02)10))
std
(160)
VF
Fi
Let the radius error and the p-level probabilistic half-width obtained at the utterance-level using average pooling equal, respectively:
B.H.P Laverag
(161)
Fi
f=1
1/2
Ft
h A.J.P Lavernge
Vg +1
(162)
VF
F
For the PESQ-like aggregation, let us denote its aggregation function from (49 as:
s(u) = 0.999 + 4(1 + exp(-1.3660 3.8224))
(163)
Let W be the window and H the hop of frame used for aggregation, then M' is the maximal number of possible windows. By norm submultiplicatiitty and the mean-value theorem (Hom & Johnson 2012 Sec. 5.6):
1/2
1.P pess
CoL
F Σ
(1gap)
2
Os
(164)
VM
Ou
1/2
1...P
COL VM
1 F
(oghin))
Ös
(165)
Ou.
where Cot. = [/H and by construction 0s/Ou S 1. 1.3669 when evaluated at point u.
To translate utterance-level errors source-based PCC and SRCC values, let the integration of utterance-level MOS ratings from every system 9 E (1,...,Q) be:
MOs
Q.1 M.Ms)
(166)
and similarly, denoting agd.P as the estimated aggregated measure across systems, where A is either average or PESQ-like aggregation B.4), then its integration is given by:
Q.P.P. ViAA
(167)
For every vector v, denote its centered version by v. Let us denote the PCC value between an observation vector v and a MOS vector m as PCC (v, m). according to (53) and 54). Its gradient
with respect to v at point I.P. A is Benesty et al. 2009):
PCC
PCC
I.P. i,d
v
VYMOs
MOS
Pl.P i.A.
(168)
Ov
1.A
i.MOS
i.A
41

---

Published as a conference paper at ICLR 2026
Consider b A the utterance-lewe level bias radii from (611 or 164 across all systems:
LP A.A
(169)
Then, the induced PCC bias can be bounded by:
PCC av I. i
P.P.POC A
(170)
For the probabilistic half-width, we model independent Gaussian jitters across systems with scales fixed by the utterance half-widths, Consider the Q- Q-dimensional Gaussian vector:
Nr(O,
Q.I.P h
2
n N 0, diag
LA
(171)
Z
Z
with 0 E R Using the delta method, first-order error propagation gives:
T diag
1,1. MiA
P i.A
2
Or
PCC 2c- A
(172)
001
Zet
Zet
Turning to the SRCC, let p (SRCC(,,)) denote Spearman's rank correlation between two vectors (Kendall & Gibbons 1990), as defined in (55) and (56). Because ranks are piecewise-constan a safe deterministic error radius is obtained by checking the two extreme bias orientations:
P.P.SCC A.A
max
SRCC
(VLP P.A
+
b.P.
A, V.,Mss) - SRRCC (V 1,A v MOM
(173)
SRCC
P.. 1.A
bIP v! i. vi,s p SRCC (pI.P L, vos
J.P For the probabilistic walf-width we jitter E.A with the same independent Gaussian model in and report the empirical e* quantile from Monte Carlo of the following:
171
h
LRRCC 1
Quantile
(PERCO
7, vi,os
SROC
(V v.
(.MOs)
(174)
where we used 1 draws for estimation, in the spirit of quantile bootstrap Tibshirani & Efron 1993).
Lastly, we consider the error propagation across all trials and their sources in given scenario, e.g.. English mixtures. Let L denote the number of trials in scenario, and for each trial E {1,..., L}. assume number of total speakers trials and NZ speakers, following 58 -8..1.
trial is N max (57). The values we report average across
The deterministic error radius of the PCC and SRCC per scenario are respectively given by:
N PCC.
PCC
(175)
E1=1
Ni
max
S.A
BSRC
ΣΣ
I.P. SRCC
(176)
NI max
To yield the probabilistic term, assume that within any fixed trial I, the pairwise correlation between the source jitters has been empirically estimated and denoted by Pt. while jitters from different trials are independent. This assumption holds by the construction of our trials in every
42

---

Published as a conference paper at ICLR 2026
scenario. Consequently, the c-level probabilisticc on the scenario mean equals:
1000
(177)
Ni
P.C.PC
N DUs
h
J.P.POC
PCC
V.A
+ 2
Σ
>
N
Vmxx
1=1
i=1
Zc
2c
1,SCC
(178)
N
L.P.SRCC
NA
L.P.SRCC A
SRCC
%6
+ 2
21=1
N
1=1
i=1
Ze*
f,==1
Z
ZC*
i<}
Ultimately, for each scenario and each measure P that uses aggregation technique A, we report the deterministic envelope and probabilistic half-width jPCC and 1 PCC for PCC values and &SSCCC and &,SCCC for SRCC values.
G FURTHER DISCUSSIONS
G.1 LIMITATIONS
Our validation depends exclusively on the SEBASS database, the only public corpus that provides human ratings for source separation systems, which limits the diversity in acoustic and linguistic traits that multiple dataset usually carry together. Moreover, the listening tests in SEBASS ask the human raters a generic quality question, rather than questions that isolate leakage versus self-distortion. This design choice may attenuate the ground-trute sensitivity to the specific error modes that PS and PM are intended to disentangle, and can introduce systematic bias that even multi-rater averaging cannot fully cancel. Another noticeable limitation of this research concerns the aggregation techniques we employ to convert frame-level utterance-level scores. Since neither granular human ratings exist nor is there any documented data-driven mapping from granular to global human ratings, we limit the capability of the PS and PM measures by merely approximating aggregation functions.
On single NVIDIA A6000 GPU paired with 32 CPU cores with 64 GB of memory, our implementa- tion achieves a real-time factor of 1.2, e.g., when analyzing a 25 ms frame in 30 ms on average. While this enables offline evaluation and hyper-parameter sweeps, it falls short of strict real- tile-imme monitoring and may limit large-scale neural-are archhieccheetre searches and limit using the PS and PM measures inside loss function during training sessions. Profiling reveals that the dominant bottlenecks diffusion-map eigendecompositions and repeated Mahalanobis distance computations with per-frame covariance estimation for all distortions points in every cluster. We plan introduce more efficient implementations as we maintain our code repository.
We also point out that in music mixtures, 0.5% frames exhibit for the PM measure an error radius that exceeds 1, rendering these observations irrelevant. These cases should be ignored completely, and future work that focuses on the separation of music sources will further investigate this phenomenon.
G.2 POSITIONING OUR WORK AS CATALYST
The absence of large, diverse datasets annotated with fine-grained human scores remains a critical gap source separation research. We argue that introducing perceptually grounded measures is precisely what enables this gap to be closed. By releasing PS and PM as open-source tools, we provide the community with foundation on which richer benchmark datasets can be built, rather than waiting for such datasets to exist before new measures are introduced. Their availability can catalyze the creation of corpora that include human annotations at both frame-level and utterance- resolutions. Such resources would support systematic, fine- enninndde comparisons between objective measures and human perception, stimulate the development of new evaluation metrics and systems, and allow researchers study the relationship between granular and global ratings, aspect currently reduced heuristic
43

---

Published as a conference paper at ICLR 2026
aggregation. In this way, PS and PM act gateway toward more rigorous and perceptually aligned evaluation standards source separation.
H LLM USAGE
We used a large language model (LLM) as purppss assistant in three ways:
1. Language polishing improve clarity. Every word was read and proofed by the authors.
2. Exploration of literature. All cited literature validated by the authors.
3. Coding assistance. All code was reviewed, rewritten as needed, and tested by the authors before use.
We did delegate authorship decisions or scientific claims to the LLM. We manually verified all content, checked citations, and validated all results. No confidential or identity-revealigg information was provided to the LLM,, and use complied with dataset licenses and the ICLR Code of Ethics. We disclose this usage here as recommended by ICLR-2026. We also disclose LLM usage in the submission form. The authors take full responsibility for the submission.
44


Reviews:
[{"review_id": "DvDLucGnsL", "reviewer": "Reviewer_WKee", "summary": "This paper introduces two frame-level, intrusive metrics: Perceptual Separation and Perceptual Match, for the assessment of audio source separation systems. The approach decomposes perceptual errors into leakage and self-distortion by encoding distorted signal variants through a pre-trained self-supervised model and embedding the resulting features onto a manifold using diffusion maps. Experimental results on the SEBASS benchmark demonstrate that both metrics offer robust correlation with human MOS compared to 14 existing measures, and the paper derives error and confidence bounds for both metrics.", "questions": "1. Can the authors empirically or theoretically justify the assumption that diffusion map Euclidean distances on the perceptual manifold directly correspond to perceptual similarity, especially for out-of-domain distortions?\n\n2. How robust are the PS and PM measures to errors in the choice of self-supervised model?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The PS and PM measures distinctly address leakage and self-distortion, providing interpretable, complementary diagnostics as established in both the main text and Figure 1. This is a clear improvement over single-score evaluation methods, which frequently conflate distinct error sources.\n\n2. The derivations for both measures are grounded in Mahalanobis distance computation on a perceptual manifold, with clear steps and full error analysis.\n\n3. The paper goes beyond point estimates, deriving deterministic and probabilistic error bounds.", "weaknesses": "1. Several directly related and highly contemporary works on perceptual audio metrics, unsupervised perceptual embeddings, and manifold learning for audio are not discussed, despite their clear relevance.\n\n[1] DeePAQ: A Perceptual Audio Quality Metric Based On Foundational Models and Weakly Supervised Learning\n[2] NOMAD: Unsupervised Learning of Perceptual Embeddings for Speech Enhancement and Non-matching Reference Audio Quality Assessment\n\n2. The paper states the hypothesis that diffusion distances on the perceptual manifold align with perceived similarity, but aside from empirical validation, this foundational assumption is not theoretically justified.\n\n3. Despite benchmarking against many baselines, the evaluation gives limited direct insight into when/why PS or PM fails relative to credible alternatives (e.g., PESQ, STOI). The qualitative and error-case analysis is not sufficiently granular.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZIXbWG1Fj9", "reviewer": "Reviewer_itJ3", "summary": "The paper proposes two new measures of audio source separation quality, called Perceptual Separation (PS) and Perceptual Match (PM), that respectively aim to quantify the amount of\n- \"leakage\": to what extent an estimated source is closer to (distorted versions of) the corresponding clean source than to (distorted versions of) the other clean sources\n- \"self-distortion\": how close is the estimated source from the clean one up to \"standard\" distortions\nThe approach is primarily frame-based and relies on the knowledge of reference clean sources, which are data-augmented using various \"standard\" distortions. \n\nFor a given frame index, the time frame of each clean, augmented, and estimated source is first mapped from the waveform domain to an embedding domain using a pre-trained embedding (wav2vec 2.0 based, or MERT based, using hand-picked intermediate layers). An additional spectral-like embedding (diffusion map -based) is then computed for each source, and used to define two Mahalanobis metrics and two representatives (a centroid, and the spectral embedding of the reference clean time frame) for each source. The distance between the spectrally embedded *estimated* time-frame of a target source and these representatives, measured with these Mahalanobis metrics, then yields quantities that are aggregated to provide the final frame-based measures for each source. \nEventually, frame-based measures are aggregated over a whole signal either by averaging (for PS) or using a \"perceptually weighted scheme inspired by PESQ\" not described in the main text (for PM).\n\nAfter conducting an error analysis, the authors evaluate the proposed measures using the SEBASS database, which provides MUSHRA human ratings over a collection of signals separated using various algorithms, gathered from pre-existing source separation evaluation campaigns. The proposed measures are compared to existing quality measures using two indices : the Pearson product-moment correlation coefficient (PCC), and Spearman's rank-order correlation coefficient (SRCC). In most of the considered settings, the proposed measures achieve the best PCC/SRCC scores (or scores comparable to the best ones).", "questions": "- How does the diffusion map step with $t=1$ and $\\alpha=1$ relate to Laplacian eigenmaps or a related simple spectral embedding approach ?\n- Do you have evidence that the proposed measures \"disentangle\" leakage from \"self-distortion\" ?\n- What is the \"waveform-only\" variant of PS/PM ?\n- What is the aim and significance of the \"error analysis\" ?\n- Did you conduct new listening test with 15 raters ?\n- How were the parameters tuned ? Was there trial and error ?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "Performance measures reflecting human ratings are very helpful to help design better algorithms. In the specific case of audio source separation, \nTable 1 illustrates well that existing measures can have limited correlation with human scores from the SEBASS database, a situation that is significantly improved by the proposed measures. \nThe approach seems sensible, as it combines a first level of \"generic\" audio features from a pre-trained embedding, a second level of embedding that somehow learns the impact of \"standard\" distortions in a fine-grained way (source-specific and time-frame specific). Its soundness is confirmed by the experimental evaluation, which can in fact seem surprisingly good for an approach where no human rating is explicitly used to train the designed measures (although given the number of parameters of the method, their selection with \"cherry picking\" can be interpreted as supervised training). \nFor these reasons I rate the contribution to 3.", "weaknesses": "The paper seems to have been written very hastily, with e.g. L075 \"*Two lines about appendices and disclaimers on LLM usage*\"\n\nAs a result, the abstract is hard to grasp due to the choice of wording. For example,  the notion of \"intrusive approach\" is certainly not familiar to the average ICLR reader, and it would be better to write more explicitly that the original sources used to generate the mixture need to be available for the evaluation. The notions of \"system output\" and \"non-attributed references\" were also not immediately clear, I only guessed its meaning after reading the rest of the paper. The notion of \"granular optimization\" is also quite cryptic (as well as further usage of the word \"granular\" in the main text).\n \nEven more importantly, the paper misses a clear overview of the overall computing pipeline corresponding to the proposed measure, and I had to make my own drawings to provide the (hopefully correct) summary above. Combined with the fact that the treatment of the different sections is significantly unbalanced,  this makes the paper quite hard to read (hence my score 1 on the presentation).\n\nSection 2 is very short, while one would expect to have there an overview of the overall context and pipeline from a high-level perspective. This would seem to be the right place to highlight that the approach is frame-based, and that the frame index will be dropped later on. I missed it at first read.\nSection 3 is essentially a reminder on diffusion maps, in my opinion this would have been better fit to an appendix : isn't it enough to know that it replaces a collection of vectors by a spectral-like embedding of the said vectors ? By the way I believe that it is an abuse of notation to consider $\\Psi_t$ (or $\\Psi_t^{(d)}$) as a function defined on $\\mathbb{R}^D$: is only defined at the considered $x_i$'s, there is no natural out-of-sample extension for an arbitrary $x \\in \\mathbb{R}^D$. \nSuggestion: since the time-frames are denoted y_*, their (e.g. word2vec) embedding $x_*$, it would lighten the notations and ease the reading to use another letter, e.g. $v_*$ to denote what is currently written $\\Psi^{(d)}_t(x_*)$. It would be sufficient to say that the definition of $v_*$ depends on some parameters ($t$, $d$ or $\\tau$, but also $\\alpha$)\nGiven the choice of $t=1$ in all experiments, the approach seems to fall back on a simpler method without diffusion, which seems to be very close to a plain Laplacian eigenmaps.\n\nSection 4 is clear, but in my opinion would gain much by reworking the notations as suggested above. \n\nThe claim about the limits of existing measures L042 (\"defined with an intrinsic ambiguity) does not seem supported by evidence, and seems in contradiction for example with the very definition of SIR/SAR/SDR.\nTable 1 does provide some evidence that the new measures \"better correlate with human global quality assessment\", yet this does not per se support a better \n\nThe notion of intrusiveness seems to mean in the introduction that clean reference sources are required. I could not relate it to the wording \"mildly-intrusive distortions\". \n\nThe notations are somewhat unclear, balancing between being too allusive or too explicit. For example, the problem formulation is frame-based, involving an index $f$ (which by the way in a signal-processing context often indicates frequency, I found this confusing at first read), but the indices are then \"repurposed\" and $f$ is dropped, this is also confusing.\n\nThe definition of some acronyms (MOS, NMI) should be recalled when they are first used several pages after being introduced.\n\n\nThere is no explanation on the definition of the \"waveform-only\" variant of PS/PM.\n\nThe aim and significance of the whole Section 5 on \"error analysis\" is totally unclear (hence my score 2 on soundness), and this reads as a series of tedious but relatively straightforward computations.\n\nAlthough the results shown in Table 1 seem to provide evidence of the interest of the proposed measures, underlining the scores on the first two lines seems superfluous.\nThe database description of Section 6.1 is ambiguous : at first I understood the SEBASS precisely provides human ratings. Then L414 seems to suggest that 15 new certified raters were involved. Were they the 15 certified raters from SEBASS ? Can this be clarified, traced and referenced ?\nIf this paper involved new certified raters, this should be reflected in the Ethics part.\n\nMy understanding is that MUSHRA and MOS are different approaches to obtain statistically significant human ratings. The authors however seem to use both acronyms as synonyms.\n\nThe values of many parameters ($L$, $\\alpha$, $t$, $d/\\tau$, $M$) and some choices (e.g., which layers of the wav2vec / MERT models are used to generate the first embedding)\nare provided without saying much on how they were selected, and whether this involved trial and error. Can you explain ?\nIf trial and error was conducted, some mild level of learning to match human ratings has been conducted, and this should be made explicit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R401PiWrgt", "reviewer": "Reviewer_h8Us", "summary": "The paper proposes two new perceptual evaluation metrics for blind audio source separation, called Perceptual Match (PM) and Perceptual Separation (PS).\n\nPM is meant to measure self-distortion: how much the separated output of a given source has been degraded (i.e. does it still \"sound like me,\" or did the separator damage me?).\n\nPS is meant to measure leakage: how much other sources are still audible in the separated output for the target source (i.e. how much bleed from others remains?).\n\nHow it works:\n\nFor each ground-truth source, the authors generate a bank of controlled distortions (filtering, gating, etc.) that are supposed to mimic plausible separator artifacts.\n\nAll short frames (tens of ms) from the clean source, its distorted variants, and the separator output are embedded into a perceptual manifold. This manifold is built with self-supervised audio embeddings (wav2vec2-like) followed by diffusion maps.\n\nEach source becomes a cluster in that manifold, with a mean and covariance.\n\nPM: does the separated output fall inside its OWN source's cluster? (High PM = low self-distortion.)\n\nPS: is the separated output closer to ITS OWN cluster than to ANY OTHER source's cluster? (High PS = low leakage.)\n\nPM/PS are computed framewise and then aggregated to clip level.\n\nThe authors evaluate on SEBASS (>11k human ratings over ~900 clips, 32 systems, English speech, Spanish speech, and music). They report that PM and PS correlate well with MOS (Pearson and Spearman), often outperforming SDR/SIR/SAR, PESQ/STOI-style metrics, DNSMOS, etc.\n\nThe paper also gives per-frame uncertainty for PM/PS: (i) a deterministic bias radius due to manifold truncation and (ii) a high-probability CI due to limited distortion samples. They propagate this to the final MOS correlation numbers.", "questions": "Q1. You claim PM and PS are the first metrics to disentangle leakage and self-distortion, but classical BSS Eval already uses SIR (interference from other sources) and SAR (artifacts). Can you give a concrete example (qualitative or quantitative) where SIR says “good” but PS clearly identifies audible leakage, or where SAR says “good” but PM clearly identifies audible self-distortion? Without that, it’s hard to judge whether PM/PS are fundamentally new or essentially SIR/SAR in a learned perceptual space.\n\nQ2. For SEBASS speech mixtures, you evaluate only English-only and Spanish-only overlaps and justify this by saying “realistic conversations are monolingual.” In many deployment settings (code-switching, bilingual overlap), that assumption does not hold. How would PS behave on overlapping multilingual speech, where two simultaneously active speakers may differ by identity but not cleanly by language? Is PS partly exploiting language identity to discriminate sources?\n\nQ3. PM is averaged over frames, but PS is pooled using a “PESQ-like perceptual weighting,” which is not explicitly defined in the paper. Can you provide the exact temporal pooling rule you use for PS? Right now PS is not reproducible from the description, and your correlation claims with MOS depend on that pooling.\n\nQ4. PM and PS depend on a hand-crafted distortion bank that defines the “perceptual neighborhood” of each source. How sensitive are your metrics to the choice of those distortions? For example, if we remove one distortion type or add an unrealistic one, do PM/PS scores or MOS correlations change significantly? This matters for whether others can safely reuse your metric or will overfit to your particular bank.\n\nQ5. The perceptual manifold uses SSL embeddings + diffusion maps with α = 1, diffusion time t = 1, and dimensionality chosen by a 99% spectral-mass rule (≈20–40 dims). Were any of these hyperparameters, or the PS pooling weights, tuned using SEBASS MOS labels (even indirectly)? If yes, how did you prevent test-set leakage when reporting final Pearson/Spearman correlations with MOS?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. Clear problem framing.\nThe paper separates two real failure modes in source separation: \"you damaged my target\" vs \"you left the other source in.\"\nThis is actually how practitioners think about failure, but most metrics don't expose it.\n\nS3. Human-grounded evaluation.\nThey compare directly to listening scores (SEBASS), not just SDR.\nThey include English speech, Spanish speech, and music (with and without drums), and 32 different separators.\n\nS4. Uncertainty / confidence intervals.\nSection 5  gives an error band on scores and pushes that through to MOS correlation.\n\n\nS5. Actionability.\nPM and PS are frame-level before pooling. This helps to point where leakage happened\" or \"this is where I over-gated the vocal.\"\nThat's a lot more usable than one global SDR number.", "weaknesses": "W1. Novelty vs SDR/SIR/SAR is under-explained.\nThe paper says PM/PS are the first metrics to \"functionally disentangle\" leakage and self-distortion.\nBut classical BSS Eval already has SIR (interference from other sources) and SAR (artifacts).\nThe paper mentions SDR/SIR/SAR but does not clearly argue why PS is NOT just SIR in a fancier embedding,\n\nwhy PM is NOT just SAR in a fancier embedding.\nWithout that, some readers will see this as repackaged SIR/SAR.\n\nW2. Assumptions in the distortion bank.\nThe method assumes that the authors' hand-designed distortion bank actually spans \"plausible\" separator artifacts.\nIf the bank misses common artifacts (phasiness, musical noise, transient smearing), PM could wrongly accept bad audio as \"still me,\" or wrongly reject good audio that falls outside their curated distortion styles. There is no ablation/sensitivity analysis on which distortions are used, so robustness is not proven.\n\nW3. Monolingual-only evaluation in speech.\nFor speech, they say they split SEBASS mixtures into English-only and Spanish-only pairs \"because realistic conversations are monolingual.\" That is not generally true (code-switching and bilingual overlap are routine in real deployments).\nThis matters because PS is partly \"do you sound like the other source?\"\nIf the other source is always a different language, that discrimination problem is easier.\nSo PS may look stronger than it would in truly multilingual overlap.\nThis should be acknowledged as a limitation.\n\nW4. PS pooling over time is not fully specified.\nPM is averaged over frames.\nPS is said to use a PESQ-like perceptual weighting so that short, audible bleed is penalized more.\nBut the exact weighting rule is not written in the main text.\nWithout that, PS is not actually reproducible from the description.\n\n\n\nW5. External validity.\nAll clips are ~10 seconds, 16 kHz, fairly clean, from curated mixtures.\nThere is no far-field mic, noisy phone audio, long meetings, reverberation, etc.\nThat is exactly the setting where applied systems struggle.\nSo the claim is really \"PM/PS correlate with MOS on SEBASS-like curated data,\" not \"PM/PS solve perceptual eval in the wild.\"\nThe paper should say that clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)