Score-based Source Separation with Applications to Digital Communication Signals
Authors: Tejas Jayashankar
Gary C.F. Lee
Alejandro Lancho
Amir Weiss
Yury Polyanskiy
Gregory Wornell
Venue: NeurIPS 2023 poster (2023)
Keywords: Diffusion models
score-based models
source separation
digital communications
maximum a posteriori (MAP) estimation
alpha-posterior
Gaussian smoothing
score distillation sampling
radio frequency systems
interference mitigation
TL;DR: We propose alpha-RGS, a novel score/diffusion-based source separation method leveraging generalized Bayes’ and randomized Gaussian smoothing, focusing on sources with underlying discrete nature, with applications to digital communication systems.

Abstract: We propose a new method for separating superimposed sources using diffusion-based generative models.  Our method relies only on separately trained statistical priors of independent sources to establish a new objective function guided by $\textit{maximum a posteriori}$ estimation with an $\textit{$\alpha$-posterior}$, across multiple levels of Gaussian smoothing.  Motivated by applications in radio-frequency (RF) systems, we are interested in sources with underlying discrete nature and the recovery of encoded bits from a signal of interest, as measured by the bit error rate (BER). Experimental results with RF mixtures demonstrate that our method results in a BER reduction of 95\%  over classical and existing learning-based methods.  Our analysis demonstrates that our proposed method yields solutions that asymptotically approach the modes of an underlying discrete distribution. Furthermore, our method can be viewed as a multi-source extension to the recently proposed score distillation sampling scheme, shedding additional light on its use beyond conditional sampling. The project webpage is available at https://alpha-rgs.github.io.

Score-based Source Separation with Applications to Digital Communication Signals
Tejas Jayashankar' Gary C.F. Leel
Alejandro Lancho 1.2 Amir Weiss
Yury Polyanskiy'
Gregory W. Wornell
*Massachusetts Institute of Technology Universidad Carlos III de Madrid
Abstract
We propose a new method for separating superimposed sources using diffusion- based generative models. Our method relies only on separately trained statistical priors of independent sources to establish a new objective function guided by maximum posteriori estimation with an o-posserior, across multiple levels of Gaussian smoothing. Motivated by applications in aadi-ffreueecy (RF) systems, we are interested in sources with underlying discrete nature and the recovery of encoded bits from a signal of interest, as measured by the bit error rate (BER). Experimental results with RF mixtures demonstrate that our method results in a BER reduction of 95% over classical and existing learning-based methods. Our analysis demonstrates that our proposed method yields solutions that asymptotically approach the modes of an underlying discrete distribution. Furthermore, our method can be viewed as multi-souree extension to the recently proposed score distillation sampling scheme, shedding additional light on its use beyond conditional sampling. The project webpage is available at https: http//lllaaaaraeeeeeeeeeeaa rgs. gi github.ir. io.
Introduction
The problem of single-channel source separation (SCSS) is ubiquitous and arises in many different applications, ranging from the cocktail party problem in the audio domain to interference mitigation in the digital communications domain. Broadly speaking, the goal SCSS decompose a mixture K1X1 + + KR E yN into its K constituent components, {x, Hx E X.N. Motivated by applications in modern engineering, e.g., intelligent communication systems, we are particularly interested in the SCSS problem for heterogeneous sources with underlying discrete structure.
Prior that deep learning for source separation problems has been well-studied in the audio and image domain by leveraging domain-specinn structures. For example, recent end-to-end methods in the audio domain leverage spectrogram masking techniques separate sparse time-frequency representations [1, 2]. while in the visual domain, natural images may be separable by exploiting local features and "smoothness" assumptions in the color space [3, 4]. More recent efforts have tried to solve this problem by leveraging independently trained statistical priors using annealed Langevin dynamics sampling [5-7]. However, these methods have demonstrated shortcomings on discrete sources that exhibit intricate temporal structures with equiprobable multimodal distributions [8].
These additional challenges motivate to develop novel general Bayesian framework for source separation that relies on statistical priors over sources with underlying discreteness. Specifically, we focus on learning data-driven priors for two main reasons-J Unknown system parameters, e.g., the underlying signal generation model may not be available to create hand-crafted priors; and ii) Automation, facilitated by learning methods create plug-and-play priors for versatile use.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

In this paper, we study the SCSS problem involving a mixture of two signals, s and b, which interfere different relative scale to produce the mixture signal y + kb, where x E R, the scaling coefficient. We additionally impose restrictions on access to paired training data samples (s, b, during training. Our motivation for this restriction arises from the following consideration; given n. sources, the number of two-component source separation models leveraging joint-statistics grows as (n²)) Any changes to the training data-even for a single source-word updates. In contrast, solutions that leverage independently trained priors over the sources need to update only a single model (i.e., (1)). Additionally, these priors can also be used to solve more generalized source- separatttn problems, e.g.. with K > 2 components or with different mixture model, without requiring any additional training.
require O(n) model
We demonstrate the successful operation of our method in the digital radio- fuetu-freyuecyy (RF) setting. Signals in the RF domain are often discrete in nature as their randomness arises from bits/symbols that modulate continuous waveform. As such, source separation performance is measured not only through continuous reconstruction metrics, e-g., the mean squared error (MSE), but also (and primarily) by the fidelity of digital communication, commonly measured by the bit error rate (BER).
Broader Impact. The crux of our work is motivated by challenges arising in the next generation of wireless systems. The proliferation of intelligent wireless Devices-utilizinn finite spectrum resources- has called for better interoperability strategics, so as to ensure reliable operations in a rapidly growing heterogeneous wireless networking ecosystem [9-11]. Interference from other sources operating in the same channel, c.g., 5G waveforms or WiFi signals, can lead to deterioration in quality of service. While conventional approaches treat such interference Gaussian noise, there could be significant performance gains if we were able to learn and exploit intricate statistical structures of these co-channel signals [12]. We take important step towards this goal by developing data- driven score-based SCSS solutions that demonstrate significant gains over modern and conventional approaches, thereby unveiling novel intelligent and effective interference mitigation strategies. We further elaborate on the broader impact of our work in Appendix A.
Contributions. We start from first principles and propose new Bayesian-inspired method for SCSS. Our main contributions are as follows.
We present a new method for SCSS that leverages the (approximate) score from pre-trained diffusion models to extend MAP estimation using generalized Bayes' theorem with an a-posterior [13, 14] across different levels of Gaussian smoothing. Our method, termed as a-RGS (a- posterior with Randomized Gaussian Smoothing) (see $3), is easy to implement by randomizing the predetermined training noise levels without cumbersome tuning of special annealing noise schedule, as in wengeeiinddyamics-haaae
approaches [5].
We show that our formulation is a multi-source extension of the recently proposed score distillation sampling (SDS) objective [15]. Our analysis (see $4) shows that despite randomizing across multiple noise levels, the local extrema of our loss (and hence of SDS) asymptotically approach the modes of the underlying unsmoothened source distribution(s) (see 13.2.1).
We demonstrate through experimental results (see $5) that a-RGS outperforms classical signal processing and annealed Langevin-dynamicc-e a 96% and 94.5% improvement in BER (averaged across different levels of interference) over traditional and existing dearning-based methods, respectively. While score-based methods have been adopted solve problems such as symbol detection [16] and channel estimation [17], the best of our knowledge, this is the first work that applies score-based models for the task of source separation in the RF domain.
approaches for RF source separation, with
Paper Organization. The paper is organized as follows. The necessary prerequisites are introduced in $2. $3 meticulously develops our proposed method, starting from basic principles. We provide analytical characterizations of our method in $4, with additional details available in Appendix B. In $5, we introduce our problem in the context of digital communication signals and describe our experiments along with results. A detailed summary of our results is available in Appendix G. A primer on digital communication signals and details about our datasets are available Appendix C. Details related to diffusion models in the RF context, primer on conventional baselines, and description of score-based separation baselines can be found in Appendices D, and F, respectively. We conclude with $6, where we comment on future work and on the broader impact of our method.

---

2 Prerequisites
2.1 Finite Alphabet Signal Processing
In many engineering systems, signals commonly exhibit continuous magnitudes. However, in particular cases, these signals may possess discrete properties, leading to a finite (but possibly large) number of possible realizations. Such signals are often expressed as,
(1) = Σ
Cp 9p tp))
where Cp E S are discrete symbols drawn from finite set S C C (or R) and gp(.) is continuous filter that "carries" contributions from the symbols. For example, optical or RF communications, the symbols could correspond to complex-valued mappings of the underlying bits [18), while in the discrete tomography domain, the symbols might correspond to measurements obtained from different materials with finite number of phases or absorption values [19]. In this work, will address the challenges associated with finding the global optimum within the optimization landscape formulated for separating superposition of such discrete sources.
2.2 Diffusion Models
Diffusion models are a class of generative models introduced by Sohl-Dickstein al. [20] based on the principles of thermodynamic diffusion. The model is parametrized by two Markov chains of length T-i) the forward chain 7(x xx-)), which noise gradually added to the data Xo such that at the end of the forward process, ((x)) (xx:; 0, I); and ii) the learned reverse chain po (x , in which noise is gradually removed (i.c., denoising) in order to generate new samples.
These models are trained by minimizing variational lower bound on the log-likelihood Recent work by Ho al. [21] in the image domain shows that this optimization problem translates learning denoiser To (Xe, t) parametrized as neural network with weights á that estimates the noise term in x, = Xo + V1 O2 2 ~ NO, I) across different timesteps ~ Unif ((1, ((1,..,,7)),
arg min Ex,,, [x)r () - z 2|3]
where (1- - a; JT" defines an increasing signal variance-presserving noise schedule with 0 S a; 1. The works in [21-23] show that objective (2) is equivalent to minimizing a loss whose minimizer is an estimate of the marginal score Sx, (xx) Vx, log q(x) such that,
Sz (x,) ~ -(1 -2r0
Other works have extended diffusion models to continuous time using stochastic differential equations [23], while others have used them for signals in the audio domain [24] and forecasting problems [25].
3 A Bayesian Framework for Source Separation
We now develop the Bayesian framework underlying our proposed source separation method. We are particularly motivated in developing a generalized framework that relies on statistical priors only, without any other form of domain-specific regularization.
3.1 Maximum a Posteriori (MAP) Source Separation
Signal statistical structure. Let s E S c Cd and b E Cd be two statistically independent complex- valued vector sources, where S is countable set of all realizations We assume that s has PMF P,. and we let b be an arbitrary source (potentially discrete with some noise) with PDF pb- We assume that the latter distributions are multimodal, where the probability is generally (close to) zero except at the modes. Additionally, we seek to be robust to the challenging setting where the distributions have multiple equiprobable modes, so as to develop novel methods that can tackle the finite alphabet source separation problem as is motivated in 1 We emphasize that these assumptions do not completely characterize the often complicated fine-grained statistical source structure, and we therefore rely on generative models to learn such unknown structures from data.
3

---

MAP problem formulation. Consider mixture composed of two superimposed sources,
y S + xb,
where & E R,, is a relative scaling coefficient between the two signals.
Given y and assuming & is known, in order to separate the sources, it is sufficient to estimate s since b b= (y s)/k. The MAP estimate of is then given by,
arg max (sly) sES st y=stob
(5)
Using Bayes' theorem, (5) can be equivalently expressed as, arg max Paly ys arg max Py|s(y|s)P3{s)
(6a)
SES st y=stab
aES at y=s+ab
arg min log (s(s) log pb ((y ((y-))/K), s)/R), #ES
(6b)
where the likelihood Py|s(y|s) = Pb ((y 5) /K) under the constraint (4), and we convert to negative log probabilities in (6b). Due to the underlying discreteness of s, and the potential underlying discreteness of b as well, the objective function (6b) is not differentiable. Hence, in its current form, gradient-based techniques cannot be used to solve this problem, and we must instead resort to combinatorial methods, which are computationally infeasible even for moderate dimension size d.
3.2 Proposed Method (a-RGS)
To overcome the computational complexity of combinatorial- -bssed methods, our goal is to develop new gradient-based SCSS solutions that leverage diffusion models trained on discrete sources. To this end, we propose to use multiple levels of Gaussian smoothing with an extended MAP framework with an a-posterior, such that the optimization landscape of the resulting objective function is smoothened.
3.2.1 The Smoothing Model and a-posterior Generalized Bayes'
Surrogate distribution. One can almost perfectly approximate P. (in some well-defined sense) with the surrogate distribution where š = s + E, for E. ~ WO of - 0. While now theoretically amenable to optimization via gradient descent, the sharp modes, which constitute numerical pitfalls, often cause gradient-based methods to get stuck in local extrema.
Gaussian Smoothing Model. We adopt variance-presservnng smoothing model, with adjustable noise levels, based on the formulation of the stochastic process proposed in [21]. Let {1 a.) be a sequence of noise levels such that a; ai+1 and a; 10, Define the "smoothened sources" as
(,)s) = Va,s /1 a,,s:
(7a)
b.. (s, = you (y 8) / V1-aa
(7b)
where t, u Unif ([L.... T)) and n. Zí (0, Iz). Observe that š, and b, b,-the continuous- valued proxies for s and b, respectively- have PDFs rather than PMFs, and in particular, their PDFs have infinite support, and they are differentiable. More importantly, direct consequence is that smoothens the optimization landscape and helps in preventing gradient-based algorithms from getting stuck at spurious local minima.
Generalized Bayes' with an a-posterior. We found it useful to replace the likelihood in (6a) with distribution proportional to (y (s)",, w 1. Intuitively, under the constraint (4), this sharpens the distribution of b and gives higher weight to the modes of ps relative to the natural weighting that arises from the MAP criterion. This is beneficial, for example, when pb is more complicated and has many more modes than P..
This aforementioned reweighting has been used an implementation trick in diffusion sampling with classifier conditioning [26], but we recognize this as MAP estimation with an a-posterior' where which is expressed through the generalized Bayes' rule [13, 14, 27, 28] as,
Paly (sy(s)
l Ps(s)
(8)
pusterior
tempered likelihuod prior
la should not be confused with, and has relation to, ar from the Gaussian smoothing model.

---

Algorithm Proposed Method: a-posterior with Randomized Gaussian Smoothing (a-RGS)
1: function SEPARATION(Y, Ki N, (m)) (
D N total steps, learning rate Y) at step i
2: for i x N, do
3:
1, u ~ Uni ,I..., Th Zo 24 ~ 10,
b Sample random noise levels
4:
St
(1)
v (0)
+ /1 a,2s
D Smooth s at level 6, (7a)
5:
bu
(1) .3) = yau
y -(())
/ + v1 Cn
> Smooth b at level u, (7b)
St ((())
,b), Too (b (0l0, (0l.,)).0)
6: Za. 2h rop
P. Compute scores, (3)
7:
g(++1) = (0) 1)5
an
(2) zb)
a
(Z. 2o)
Subtract noise, (12)
au
v v1-ai aa
8: end for
9: return S = ((N), b = (y - 8)/K
10: end function
In practice, using an a-posterior has demonstrated increased learning speeds [13, 14] and could potentially help in resolving model mismatch arising from the use of approximate densities scores (rather than exact ones) during optimization via gradient descent.
Single noise level estimation loss. Let (()) O be our estimate of s.² Motivated by the form in (6b), we generalize using the smoothing (7a)-(7b) in conjunction with the a-posterior with a = w to define new single noise level estimation loss,
L.) = - -og ps. Se (0) w P6, (b.. (0.
While departing from the original MAP (6b), the newly-defined approximated loss (9) thereof facilitates gradient-bart based methods, which is key to our solution approach. Particularly, by varying the level of smoothing, (9) is more easily explored in regions between the modes via gradient descent.
3.2.2 Estimation Rule Across Multiple Noise Levels
Intuitively, larger noise variances allow us to move between modes during gradient descent. In contrast, at lower noise levels the modes sharper, which is beneficial in resolving the solution, assuming the iterative procedure starts at the basin of attraction of the correct local extremum point, or, alternatively, another "good" local extremum point. We propose new estimation rule that uses (9) across multiple noise levels. Randomizing over multiple levels of Gaussian smoothing, our proposed gradient update asymptotically (in the number of iterations) takes the form,
VeC(O) == -Er.z [Vai St. B5 (0)
Eu.z. [vanSte (ba (0.y))
(10)
E. Vo
where t, Unif (L..... T3). and the true score is defined as
Ss. S1 (0)
Vx log ps, (x) x0).(0))
(11)
and similarly for Ss...
Our proposed updates can be implemented using stochastic gradient descent as shown in Algorithm 1. We use pre-trained diffussion models ($2.2) to approximate the score when deriving the analytical score is not possible. Using (3), we relate the learned score to the denoiser available from the diffusion model, and also use zero mean noise corrected estimate,
Ez, re. S. (0),t) Rz, row s.. (0). -2,
(12)
in our updates as shown in Algorithm 1. The above rule is motivated to introduce numerical stability and reduces the variance of the updates, since the diffusion models squared value of the same error term in (2). The state-of-the tftte-a method of score distillation sampling (SDS) [15] that uses diffusion models critics, also uses the same term in their gradient update step.
trained to minimize the
'Owing to the continuous nature of the optimization landscape, 0 is in fact an estimate of 3 (see $3.2.1), and does affect BER and MSE estimates significantly.
5

---

20
MAP
MAP with
10
1100
C100
+ comtrain
-16
Two discrete sources, with infinitesimal additive noise, superimposed to produce a joint distribution with 8 equiprobable modes. An observed mixture y, imposes linear constraint in this space, Middle: Extending vanilla MAP (w = 1) to multiple noise levels still has a relatively large local minima. Right: By using x², we able accentuate the correct mode and smooth the landscape even further. Colored curves correspond to (15) evaluated with T = and = u.
Figure 1: Left:
3.3 Relation Other Works
BASIS separation. Jayaram and Thickstun [5] introduced the BASIS separation algorithm that leverages the score from generative model perform source separation using annealed Langevin dynamics. Unlike our method, the BASIS algorithm relies on specially tuned noise schedule for separation that is distinct from the diffusion model training noise schedule. We cireumvent the challenges associated with tuning such a schedule and instead re-use the pre-determined training noise schedule in randomized fashion. As such, the only parameter that we tune the learning rate. Our results ($5.3) show that we outperform BASIS in the context of RF source separation.
Score Distillation Sampling. Our work be viewed as extension of the recently proposed Dreamfusion architecture [15], which uses pre-trained image diffusion models as critics to guide the generation of novel 3D visual content. Given generated sample, g()). Score Distillation Sampling (SDS) updates the 3D object realization using gradient descent with a gradient given by.
V LSDS (Ø. S = g(0)) = E. (rs()) g + v ozz, t) -2)]] 2)]
(13)
where w(tt is a scaling related to the noise variance. Our updates contain the same gradient terms in (13) and hence it can be viewed
multi-soure extension of SDS, shedding light on its use in applications to problems beyond sampling with interactions between numerous individual priors.
Diffusion models as priors. More recently diffusion priors have been used in inverse problems such as MRI reconstruction [29, 30], image restoration/colorization [23, 31-34] and for developing general inversion methods [32, 35]. A majority of these works solve a MAP problem with domain-specific likelihood constraints, thus resembling our formulation. However, they either use annealed Langevin dynamics or noise contracting reverse diffusion process to sample from the posterior distribution, necessitating the tuning a special annealing schedule. our work, we demonstrate convergence to the same modes using a simpler scheme based randomized levels of Gaussian smoothing (see $4), The a e-posterior parameter can be interpreted as a weighting of the regularization term, thus shedding light on the potential applicability of our method to general inverse problems in the future.
4 Characterization of a-RGS
In this section, we characterize the behavior of our objective function through a simple yet intuitive example. We first present a sufficient condition for perfect signal separation in the context of discrete sources that follow the signal generation model in (1). Proposition 1.
Let s(t) and b(t) be two sources following (1) with underlying symbols or and cp respectively. Assume that the symbols obtained as,
C = 3= ((u)))/=)))
and e ((r))
(14)
where f :4 - and h VL - C are mappings from sequence of length L over the discrete alphabets 11 and respectively'. If the mapping between the discrete representation and the symbol representation of the sources unique, perfect recovery is possible.
IIn digital communications, the discrete alphabet is typically the binary alphabet, representing the underlying bits that need to be communicated, and the symbols refer the digital constellation (see 1).1)

---

Deyta Condebakon
Färing
3D0
Bauebard
0000
Cy 02
Modalator
Seperttr
Souce
Decoding
Depadel
Digital Medulttion roster cosine
Canter Sigua
Figure 2: The single-carrier digital modulation pipeline with an intelligent decoder that performs pre-processing stage of source separation. Illustrated is an example with a QPSK constellation and a root-raised cosine (RRC) pulse shaping function.
We now argue that under such conditions, our method Algorithm 1 asymptotically approaches a local extremum corresponding to the following loss function,
(()) = E2,, [og Ps, (s, (8)) w...... [logPs. (b (e.x))
(15)
To see this, let and b be two discrete sources with equiprobable modes at {-1, +1} and {=*/V20, = respectively. Figure shows the contours of the negative log joint probabil- ity, with the sources augmented with an infinitesimally small amount of Gaussian noise (see $3.2.1). An observed mixture y, adds a linear constraint in the (s, b) plane. The goal of Algorithm 1 is to pick out the constraint-satisfyin mode at (1, 20) If = 1, as shown in the middle plot, and given poor initialization ( closer 11, the gradients in this region may prevent the estimate from escaping the suboptimal local minimum. If instead, we use w x² > 1 (k 15.85), as shown in rightmost plot, the optimization landscape is better conditioned in the same (absolute) neighborhood around 0 = -1 with the mode at +1 much more accentuated. Specifically, if the algorithm is now initialized at ( 1, after enough iterations, gradient steps at larger noise levels would lead the solution towards the direction of A = Thus, on average (black curve), after enough iterations, the solution will approach +1. In constrast, Langevin-dynamie aammccsbbsedd approaches without the a-posterior weighting [5], if initialized poorly, could potentially get stuck at local extrema due to sharpening of the optimizing landscape as the level of noise decreases.
The estimate of the discrete source s can be obtained by mapping the solution returned by Algorithm 1 to the closest point (in the Euclidean sense) in the discrete alphabet S. This relies on the extremum o" of (15) being sufficiently close the desired mode of (sy:::). so that 0" can be mapped to the correct point in S with high probability. In the context of the above example, key observation in achieving this desired behavior is that the mode at (1, 2/)2 still prominent at large noise levels. Thus, randomizing across different noise levels helps balance the exploration between modes and the resolution of the estimate. This mainly requires a suitable noise level range (i.e., lower bound on (r)), to ensure that the modes are sufficiently resolved. In our experiments, we show that additional tuning required and that the training noise levels from the pre-trained diffusion models can be re-used, provided that the models have learned the source's structure sufficiently well.
more detailed characterization of our algorithm's 's convergence to the modes can be found in Appendix B. We underscore that in this work we focus on general signal types and mixtures, where perfect separability is not guaranteed and hence performance bounds are not analytically tractable.
5
Applications and Experiments
5.1 Digital Communication Signals
Digital communication pipeline. At a high level, digital communications deals with the transmis- sion of bits by modulating a so-called "carrier signal".. Groups of bits, from which the underlying discreteness of these sources originates, are first mapped symbols Gp E C via the digital con- stellation-a mapping between groups of bits and a finite set of complex-valued symbols. These symbols are subsequently aggregated via (I) form a complex-valued continuous waveform. In the RF domain, g(-) is known as the pulse shaping filter and helps limit the signal's 's bandwidth [36, Sec 4.4.3]. The constellation chosen (among other considerations) by the number of bits modulated simultaneously. Common schemes include modulating two bits at a time (Quadrature Phase Shift Keying, or QPSK), or one bit at a time (Binary Phase Shift Keying, or BPSK). Figure illustrates a representative modulation pipeline. To recover the bits at the receiver, one may adopt matched filtering (MF) [18, Sec 5.8] before the estimation of the underlying symbols, and thereafter decode them back to bits. For commonly used pulse shaping functions, such as the root-raised cosine
7

---

(RRC) shown in Figure 2, the matched filter and pulse shaping filter coincide. We refer readers to [18, 36, 37] for more thorough exposition of the topic.
Interference mitigation. Mitigating co-channel interference is a challenging problem, especially in heterogeneous networks [9-11]. 1]. If the system parameters and signal generation model are known ahead of time, one can leverage this knowledge devise hand-crafted priors. However, one often deals with interference from more complicated wireless sources, for which the signal model is unknown. In such scenarios, data-driven methods that learn priors from background recordings can be useful. Our results demonstrate that a-RGS, which leverages these priors, sets a new statc-of-the in SCSS for RF mixtures. We envision that our SCSS solution could help mitigate such interference prior to decoding the signal, as shown in the right hand (receiver) side of Figure 2.
Deep learning for RF systems. Recently deep learning methods have demonstrated the potential to reap significant gains over handcrafted model-based methods in RF applications [38, 39j. Some works have studied the problem of symbol detection [40, 41], where they assume that the channel is stationary. Other works, such as DeepSIC [42], use deep learning for interference cancellation in the multi-user setting within the same channel. In contrast, we deal with the more challenging setting of non-stationary interference, thereby requiring efficient exploitation of intricate temporal structures. While the latter works consider the superposition of independent and identically distributed sources (same technology), we assume unknown additive interference (cross technology), hard problem to solve with naïve decoding methods in the absence of explicit prior knowledge about the interference. Our problem formulation is closer recent work in [43]. However, they learn an end-to-end estimate of the signal from paired data samples. As motivated in $1, we instead assume restricted access to joint data, with focus on capturing properties of the components through independent priors.
5.2 Experimental Setup
We now detail the setup for training diffusion models on RF signals. We subsequently explain how these models are used in the implementation of our method at inference time (i.e, for separation).
RF SCSS formulation. We are interested in recovering s, the signal of interest (SOI), from a mixture y + xb, where b is assumed to be co-channel interference with unknown system parameters. We evaluate performance using two metrics-i) -i) the MSF, that measures the distortion between the estimated SOI and the ground truth; and ii) the BER of the decoded discrete represen- tation, which is obtained from the estimated SOI by extracting the underlying bits (see Appendix C). The latter measure is partieular to digital communication signals as it captures the fidelity of the estimated representation that is only partially reflected in the MSE criterion.
Datasets. We train diffusion models on different RF datasets -i) synthetic QPSK signals with RRC pulse shaping (see 15.1).17 ii) synthetic OFDM signals (BPSK and QPSK) with structure similar to IEEE 802.11 I WiFi signals; and iii) signals corresponding to "CommSignal2" from the RF Challenge [44], which contains datasets of over-the-air ahrr recorded signals. All synthetic datasets were created using the NVIDIA Sionna toolkit [45]. All datasets contain between 150k- -
500k samples and we use 90-10 10 train- validation split during training. Additional details are in Appendix C.
Diffusion model training. We adopt the Diffwave [24] architecture for our experiments, with minor changes (see Appendix D) to accommodate the complex-valued nature of our signals. Our models are trained in the waveform domain inputs of length 2560 with the real and imaginary components concatenated in the channel dimension. We train unconditional diffusion models and assume access to knowledge about the signal generation model. We use noise variance levels in the range 1.-4.00 0.72) discretized into 50 levels. We train for 500k steps with early stopping on 2 x NVIDIA 3090 GPUs. Detailed hyperparameters for our training setup provided in Appendix D.
Source separation setup. We consider three different mixtures in our experiments, all using an RRC-QPSK signal as the SOI s. The interference signal b is one of OFDM (BPSK), OFDM (QPSK) or windowed recording from the CommSignal2 dataset. Our proposed algorithm uses = x² and is initialized with the MF solution given the mixture y (see $5.1). ). Note that x. can be cquivalently described as the signal to interference ratio (SIR- == 1/x² = E, [[||]] [ EEp [
8

---

(OPSK)
12-1
2
18
RRC--OPSK OPTM (QPSKI Inseinenss
RRC-OPSK SOH Corn gaa Inorirencc
RRC-OPSK SOI GRIM (OPSS)) Inserforance
377
Figure 3: Left: Source separation results for a mixture with RRC-QPSK SOI and OFDM (QPSK) interference. All curves are obtained by averaging 400 different mixtures per SIR and using w = k². Our proposed method significantly outperforms traditional and barming-based baselines (BASIS) in terms BER and MSE across all noise levels. Middle: Similar comparisons only in BER for mixtures with OFDM (BPSK) and CommSignal2 interference, respectively. BER is slightly higher for the CommSignal2 mixture since it contains background noise that is amplified for large K. The black dotted line in the bottom figure denotes the (presumed) BER lower bound assuming the background noise is an additive white Gaussian noise. Right: BER and MSE versus W/ x² for different SIR levels. Clearly good choice in this setting, in the sense of minimum BER and MSE, is = =
We assume that K is known* and use N = 20, 000 with a cosine annealing learning rate schedule [46]. The OFDM mixtures use (7max: 7/inn) = (5e-3, 1e-6) and the CommSignal2 mixture uses (fmax, Mminn ) = (2e- 3, le- 6). Importantly, we re-use the training noise levels from the diffusion models and randomize over all but the smallest noise level resulting in a noise variance range of aT 0.72) discretized into T = 49 levels. We test performance across SIR levels ranging from - 24 dB to -3 dB ("strong interference" regime), by changing the value of k in the mixture. Each set of separation experiments was conducted on single NVIDIA V100 GPU.
(1-ac 01 1.2e-3,1
Baselines. We compare our proposed method against baselines that also leverage independent statistical structural priors over the sources. The simplest baseline, which nevertheless is still commonly used in most communication systems, is the matched filtering solution, which treats the interference as white Gaussian noise. The linear minimum mean square error (LMMSH) solution, commonly used technique for signal estimation, is another baseline that leverages (up to) second-order statistics of the underlying source distributions. More details on these methods are in Appendix E.
We also compare with the BASIS separation algorithm (see $3.3), which is the closest learning- method that resembles our problem formulation. Applying their method as is yielded poor results, and hence we modified the original hyperparamters by tuning the annealing schedule to the best of our abilities for fair comparison. For more details on this, refer to Appendix F.
To study the fidelity of our learned score models, we derive the analytical score function of the QPSK SOI in the symbol domain (i.e., before pulse shaping). We use this analytical score as another comparison to demonstrate the performance of our method the score was known perfectly. This formulation is closer to sarming-based interference mitigation setting, where we assume perfect knowledge about the SOI model, and rely on a learned interference model.
5.3 Source Separation Results
Figure 3 shows the source separation results for the three different mixture types in $5.2, obtained by averaging 100 independent trials per SIR. Our proposed method (analytical or trained SOI score) significantly outperforms MF, LMMSE and BASIS in terms of both BER and MSE. As expected, our best results are obtained by leveraging the prior knowledge, in the form of the analytical score
Maay communication systems have power constraints and equalization capabilities, and with the endowment of such knowledge it is possible to estimate the signal to interference ratio (SIR) within reasonable margin.

---

for the SOI. Nevertheless, our learned SOI score can nearly mimic this performance, and despite the slight degradation still outperforms all baselines in terms of BER as well. It should be noted that CommSignal2 contains small amount of background noise, which is amplified at low SIR (high k). This noise constrains the minimal achievable BER even under the assumption of only having the residual AWGN present, illustrated by the black dotted line at the bottom middle plot in Figure 3. We also outperform the baselines terms of MSE across all SIR levels, an example of which is shown at the bottom left of Figure 3. More detailed results are included in Appendix G.
Characterizing the choice of w. We numerically verify that w x² good choice for the a-posterior term. To this end, we find a suitable order of magnitude for w. by varying w/xx between 10-2 2 and 102 and computing the resulting BER and MSE for four representative SIR levels using held-out hyperparameter validation set of 400 mixtures. The right panel of Figure 3 shows that w = x² achieves the minimum BER and MSE on average. More characterizations are in Appendix G.
Concluding Remarks
In this work, propose a-RGS, method that extends MAP estimation with a-posterior across randomized levels of Gaussian smoothing, which stems from a new objective function, whose extrema points correspond the modes of the underlying discrete distribution of interest. Our method relies only on pre-trained diffusion models as priors via a simple randomized algorithm that does not require cumbersome tuning of special annealing schedule, as is done in existing works. Through simple analytical illustrations, we demonstrate the favorable mode mooee-preeervig nature of our objective, and show that a-RGS is generalized extension of the recently proposed SDS loss. Experiments on RF sources demonstrate superior separation performance, with gains of up to 95% in terms of both BER and MSE over classical and existing score-based SCSS methods.
based
Limitations and future work. We believe there is a potential to use our algorithm to develop general toolkit of algorithms for source separation and inverse problems. However, we do list some limitations of our method that will be a focus of future work: 1) inference time (~ 5 min per mixture) requires speed-up in real-time systems; 2) studying settings with more than two mixture components or other mixture models; 3) developing systematic approach to choose for practitioners; and 4) addressing the robustness our method for applications to signals with new properties, c.g., gravitational waves. We further elaborate on the broader impact of our work in Appendix A.
Acknowledgments and Disclosure of Funding
Research was supported, in part, by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator under Cooperative Agreement Number FA8750- 19-2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
The authors acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the research results reported within this paper.
Alejandro Lancho has received funding from the European Union's Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curfe Grant Agreement No. 101024432. This work is also supported, in part, by the National Science Foundation (NSF) under Grant No. CCF. 2131 115.
10

---

References
[1] Pritish Chandna, Marius Miron, Jordi Janer, and Emilia Gómez. Monoaural audio source separation using deep convolutional neural networks. In Latent Variable Analysis and Signal Separation: 13th International Conference, LVA/ICA 2017, Grenoble, France, February 21-23,
2017, Proceedings 13, pages 258 23-266..22 Springer, 2017.
[2] Efthymios Tzinis, Zhepei Wang. and Paris Smaragdis. SuDoRM-RF: Efficient networks for universal audio source separation. In 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1-6. IEEE, 2020.
[3] Yosef Gandelsman, Assaf Shocher, and Michal Irani. Double- DIP: unsupervised image de- composition via coupled deep-image-priors. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 11026-11035, 2019.
[4] Zhengxia Zou, Sen Lei, Tianyang Shi, Zhenwei Shi, and Jieping Ye. Deep adversarial decom- position: A unified framework for separating superimposed images. In Proceedings of the IEEE/CVE conference on computer vision and paltern recognition, pages 12806-12816, 2020.
[5] Vivek Jayaram and John Thickstun. Source separation with deep generative priors. In Interna- tional Conference on Machine Learning, pages 4724-4735. PMLR, 2020.
[6] Giorgio Mariani, Irene Tallini, Emilian Postolache, Michele Mancusi, Luca Cosmo, and Emanuele Rodolà. Multi-source diffusion models for simultaneous music generation and separation. arXiv preprint arXiv:2302.02257, 2023.
[7] Shahar Lutati, Eliya Nachmani, and Lior Wolf. Separate and diffuse: Using a pretrained diffusion model for improving source separation. arXiv preprint arXIv:2301.1072, 2023.
[8] Maurice Frank and Maximilian Ilse. Problems using deep generative models for probabilistic audio source separation. In "I Can't Believe It's Not Better!" NeurlPS 2020 workshop, 2020.
[9] Aleksandar Damnjanovic, Juan Montojo, Yongbin Wei, Tingfang Ji, Tao Luo, Madhavan Vajapeyam, Taesang Yoo, Osok Song, and Durga Malladi. survey on 3GPP heterogeneous networks. IEEE Wireless Communications, 18(3):10-21, 2011.
[10] Aamod Khandekar, Naga Bhushan, Ji Tingfang, and Vieri Vanghi. LTE-advanced: Heteroge- neous networks. In 2010 European Wireless Conference (EW), pages 978-982. IEEE, 2010.
[11] Yongjun Xu, Guan Gui, Haris Gacanin, and Fumiyuki Adachi. A survey on resource alloca- tion for 5G heterogeneous networks: Current research, future trends, and challenges. IEEE Communications Surveys & Tutorials, 23(2):668 66016666666666, 2021.
[12] Alejandro Lancho, Amir Weiss, Gary CF Lee, Jennifer Tang. Yuheng Bu, Yury Polyanskiy, and Gregory W Wornell. data-driven blind synchronization and interference rejection for digital communication signals. In 2022 IEEE Global Communications Conference (GLOBECOM), pages
2022.
[13] Lucie Perrotta. Practical calibration of the temperature parameter in gibbs posteriors. arXiv preprint arXiv:2004. 10522, 2020.
[14] Chris Holmes and Stephen G Walker. Assigning value to a power likelihood in a general Bayesian model. Biometrika, 104(2):497-503, 03 2017.
(15) Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3D using
2D diffusion. arXiv preprint arXiv:2209. /4988, 2022.
[16] Nicolas Zilberstein, Chris Dick, Rahman Doost- Moost-Mohamady,, Ashutosh Sabharwal, and Santi- ago Segarra. Annealed langevin dynamics for massive mimo detection. IEEE Transactions Wireless Communications, 2266)3376-33 2023. doi: 10. 119/TTC.2202..3220055..
[17] Marius Arvinte and Jonathan Tamir. Mimo channel estimation using score-based generative models. IEEE Transactions Wireless Communications, 2022.
[18] Amos Lapidoth. A foundation in digital communication. Cambridge University Press, 2017.
11

---

[19] Emmanuelle Gouillart, Florent Krzakala, Mare Mezard, and Lenka Zdeborová. Belief- propagation reconstruction for discrete tomography. Inverse Problems, 29(3):035003, 2013.
[20] Jascha Dick-Dicsstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-22662 2265. PMLR, 2015.
[21] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-688 6851, 2020.
[22] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances neural information processing systems, 32, 2019.
[23] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Reprexentations, 2021.
[24] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
[25] Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denois- ing diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, pages 8857-8868. PMLR, 2021.
[26] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
[27] Tong Zhang. Learning bounds for generalized family of bayesian posterior distributions. Advances in Neurul Information Processing Systems, 16, 2003.
[28] Peter Grünwald. The safe bayesian: learning the learning rate via the mixability gap. In Algorithmic Learning Theory: 23rd International Conference, 2012 Lyon, France, October 29-31, 2012. Proceedings 23, pages 169-183, Springer, 2012.
[29] Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medi- cal imaging with score-based generative models. In International Conference on Learning Representations, 2022.
[30] Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated MRI. Medical Image Analysis, 80:1024111 102479, 2022.
[31] Zahra Kadkhodaie and Bero Simoncelli. Stochastic solutions for linear inverse problems using the prior implicit in a denoiser. Advances in Neural Information Processing Systems, 34: 13242-13254, 2021.
[32] Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare L Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems, arXiv preprint arXiv:2209. 14687, 2022.
[33] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[34] Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 2022.
[35] Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guidee diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
[36] Robert W Heath Jr. Introduction to wireless digital communication: a signal processing perspective. Prentice Hall, 2017.
[37] Andrea Goldsmith. Wireless communications. Cambridge University Press, 2005.
12

---

[38] Yonina C Eldar, Andrea Goldsmith, Deniz Gündüz, and H Vincent Poor. Machine learning and wireless communications. Cambridge University Press, 2022.
[39] Taiwo Oyedare, Vijay K. Shah, Daniel J. Jakubisin, and Jeffrey H. Reed. Interference sup- pression using deep learning: Current approaches and open challenges. IEEE Access, 10: 66238-666 66236,6666, 2022. doi: D..1099AACCESS.222....111
[40] Neev Samuel, Tzvi Diskin, and Ami Wiesel. Learning to detect. IEEE Transactions on Signal Processing, (7110)2554-225555 2019. doi: 10.1009/19922092289995.
[41] Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, and Andrea J. Goldsmith. Viterbinet: A deep learning based viterbi algorithm for symbol detection. IEEE Transactions on Wireless Communications, 19(5)3319-3331 2020. doi: N0.1109/TWC2200029973322
[42] Nir Shlezinger, Rong Fu, and Yonina C. Eldar. Deep soft interference cancellation for mimo detection. In 2020 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8881 -8885, 2020. doi: 10.1 N0.100//CASS5447776.200000 9054732.
[43] Gary CF Lee, Amir Weiss, Alejandro Lancho, Jennifer Tang. Yuheng Bu, Yury Polyanskiy, and Gregory W Wornell. Exploiting temporal structures of cyclostationary signals for data-driven single-channel source separation. In 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP), pages 1-6. IEEE, 2022.
[44] MIT RLE. Single-Channe RF Challenge. challenge-1/. Accessed 2022-11-07.
URL nttps://rfchalleee wwwinngeee.........m.......m...e edu/
[45] Jakob Hoydis, Sebastian Cammerer, Fayçal Ait Aoudia, Avinash Vem, Nikolaus Binder, Guillermo Mareus, and Alexander Keller. Sionna: An open-source library for next-generation physical layer research. arXiv preprint, Mar. 2022.
[46] Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017.
[47] Herbert E. Robbins. An empirical bayes approach to statistics. 1956.
[48] Paul Bromiley. Products and convolutions of Gaussian probability density functions. Visiin Memo, 3(4):1, 2003.
[49] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances neural information processing systems, 30, 2017.
[50] David Tse and Pramod Viswanath. Fundamentals of wireless communication. Cambridge University Press, 2005.
[51] Vladimir Kulikov, Shahar Yadin, Matan Kleiner, and Tomer Michaeli. SinDDM: single image denoising diffusion model. arXiv preprint 1rX1v:2211.16582, 2022.
13

---

Supplementary Material
A Broader Impact
A.1 Regarding the Proposed a-RGS Method
Contribution and Distinction from Existing Works. Our method is in the effort of i) using score- based/diffusion based methods beyond sampling, and to solve inverse problems, and ii) developing source separation strategies with focus on sources with underlying discrete properties, which may make differ from existing works where the usual goal is to obtain perceptually close solutions, which is not good criterion in the RF domain.
Scalable Approach to Source Separation with Pre-Trained Models. An attractive property of our method that by using pre-trained models as independent priors, we minimize the need for expensive compute resources retrain models to tackle different scenarios/diifereet mixture settings.
Trade-off in Computation Time. While the training time reduced, the inference computation time is of now longer than other methods (e.g., end-to-end training with mixtures inputs and SOIs as desired outputs), thus making it unsuitable for real-time operation. As we discuss about its potential use for applications, speeding up the algorithmic aspects of the operation essential in furthering this method towards practical applications.
Ethical Considerations of our Work. The intersection between generative Al and RF signals is a nascent field and hence, their ethical implications have been less explored. Nevertheless, we believe that this an area that requires special attention to address issues such as malicious parties learning and spoofing, among others. While these are not the focus and the intent of our work, we believe that works in security this space are actively exploring such problems.
Privacy Considerations with Generative Models. Where privacy and sensitive information is concerned, our work solely focuses on synthetic and public datasets. No sensitive transmissions have been captured, and therefore learned, by our generative models.
A.2 Regarding Applications to RF Systems
Novelty, Relevance and Timeliness. To the best of our knowledge, this is the first work that uses score-based methods for physical layer transmission in digital communication systems. The interfer- ence mitigation problem has significant relevance enabling reliable operations in next-generatic wireless systems. With the growth of heterogeneous wireless networks, there is overcrowding and growing scarcity of wireless spectrum. This work explores interference mitigation solutions with the goal of enabling efficient sharing of the finite resources to allow growth of the wireless ecosystem in a sustainable way.
Proof-of-Concee Exploration. Our work demonstrates via proof-of-concept what might be possible for non-orthogonal access to explore opportunities for new generation of transmission strategies and communication protocols; particularly, the emphasis on the low SIR regime is in line with motivations in lower-power device operations.
Considerations arising from Demonstration via Computational Simulations. All our exper- iments are done via computational simulation and thus, we did not operate in RF bands that we do not have rights to access. Our synthetic simulations may differ from practice where unmodeled physical phenomenon may occur, thus necessitating natural extensions of our work to deal with such impediments.
'That is, neighboring communicating devices that operate of the same, shared physical resources (e.g.. time and frequency).
14

---

B Additional Characterizations of a-RGS
We now "dissect" (15) in $4, and focus single term,
(0) = Et.,, [log Ps, (st (0))],
(16)
where we recall the definition of the Gaussian smoothing model from $3.2.1,
S ( va vI a,Zy 2, ~ (O,
(17)
As detailed in $3.2.1, when estimating a discrete source is continuous surrogate of the discrete estimate 0, useful for optimization via gradient descent. The above loss only depends on the prior of the source, and particular independent of the data. Therefore, this term serves (and can be viewed) as regularizer in our inference optimization problem. Although analysis of (16) (strictly speaking) not useful order to make statements about (15), nevertheless informative and insightful to show that the local extrema of (16) approach the modes of the underlying source distribution Ps. by solving for the stationary points,
0 Eg for N o2 o, +
Bx, (o log Ps, (re0 + ot2,)]
(18)
where we have used Leibniz rule for differentiation under the integral and where we have defined the shorthand notation,
Va and of ===v -1=vv-aa az.
(19)
Through our examples will demonstrate the mode-approaching nature of the solutions to (18) thus establishing another interpretation of the asymptotic behavior of our method-two loss terms whose stationary points are modes of the corresponding source distributions which work together in unison to satisfy the constraints imposed by the observed mixture. We thereby end up, with high probability, in the correct mass pass-points of the underlying distributions.
B.1 Multivariate Normal Sources
We first start with the analysis of multivariate normal source and show that the the exact mode is obtained as the local extremum of (16).
Proposition 2. Let Rdd be multivariate normal source with mean sty and covariance matrix Σ,. For the Gaussian smoothing model in (17), the score at timestep is
(x)) log Ps, (St (s)) = (715 + o.Z; pg.)
(20)
Ps. = 1 and Es, = 2/ 0 Then, the minimizer 0* arg ming (0) is equal to Mh i.e., the mode of the source distribution.
Proof. Since s is a continuous source, we perform optimization over a continuous space with respect 0 Rd Notice that,
Vo log (n:0 o12,) = V (0)) log Ps, (s: (0))
(21a)
7: Dz. (n 0 + o,2, Ms,)
(21b)
Substituting (21b) in (18), we see that the minimizer must satisfy
Eg
D (7:0 m4,) = O,
(22)
where we have used the fact that 2₂ is zero mean Gaussian realization. Thus, the minimizer is
T
0"
Σ
D-1
Σ
Σ 14g
(23)
which can be simplified further as where we have used (2) to substitute Ms. 7t
0-
Bt
Y't -ー
fyy
(24)
Thus, we have established that the local extremum of the (16) corresponds to the mode in the multivariate Gaussian case.
In this continuous setting, surrogate distribution is needed and 0 =0 (17)
15

---

B.2 Tweedie's Formula and Digital RF Sources
We next analyze (16) in the context of RF source separation by studying sources defined by a digital constellation with symbols from the finite set A. Before proceeding, we first state Tweedie's formula [47]. a tool that will aid us in computing the scores of arbitrary distributions. For completeness of the exposition, we prove it for the case of scalar random variables below.
Lemma 1. (Tweedie' 's formula) Let a be random variable with an arbitrary distribution and let Za be normal random variable with mean and variance o² independent of a. The score of the random variable == a + Zo is
1 σ² - Bapx za]
Va lo)
(25)
Proof. Since x is sum of two independent random variable, it follows convolution distribution given by,
Pa(x) (Ps Pe, (x))
(26)
Therefore,
V log(ps (x)
(Pa * V=Pen))2))
(27)
(ps Pea (xx
Pa(t) Pe, Pe, (x 1) df
(28)
(ps Pz. (I)
(# (p-xx de
Pa(1)pz, (x
(29)
(pa Pz, (x) 1
1
tPalazz, (1/x) dt
(30)
0²
1
+
B[alr lr = a zo
(31)
g²
where (29) follows from,
(11 x
V Pzo (xc -2o (x 1)
(32)
r²
Proposition 3. Let be scalar random variable representing symbol drawn uniformly from the finite constellation set A C. C. For the Gaussian smoothing model in (17) where, at timestep t,
St (s) 7/88 + otžti
the score is
(B)(())) log1s, (8t (8))
- (8) + Σ a da ((()))
(33)
oo
aEA
where,
exp
(a, (()))
=
o7
(34)
Σ
BEE exp(
8.(0)
7.0
07
Proof. We will use Lemma to compute the score, for which the only quantity we need to compute the conditional expectation. Let s' = n.s. Since is discrete, Psr is also uniform distribution over
16

---

the symbol set A, = tnal /a E A]. By Bayes' rule,
Pls' + 0,2, = ==/ss'=//p/ = na) PP ss' = %
P s = nals' 0,z, = 2|
(35)
Eaes A IP sss' + 0z2g
2/55 ya P Q
P Zs
5e
T
(36)
A
IP
2g
2 nd
1 expl
2
of
(37)
Eaed exp(
12 7
Φr(a,
(38)
Therefore, from Lemma
1 ((((()) log (5x (5))
(s)+ Σ a. (a,
of
aEA
As an illustrative example, consider a BPSK source with modes -1 and +1. To solve (18) for 0, we require the score of the BPSK source different non-zero levels of Gaussian smooth- ing. Note that since the source is discrete, the score undefined if no smoothing is applied. Though this smoothened score can be computed using (33), the stationary points of (18) cannot be computed in closed-form. Hence, in order to study the behavior of the solutions, we simulate (16), as shown in Figure 4. The colored curves plot the loss for single noise level, i.e., when T= 1. while the black curve computes the loss across multiple levels of smoothing, by comput- ing average
10at
10
200
10
Figure 4: Simulating (16) on a BPSK source. The loss individual timestamps is visualized ad- dition to the total loss. The minima are at the modes of the BPSK source distribution, and +1. Larger noise levels allow for exploration be- tween modes and smaller noise levels sharpen the mode-seeking behavior.
the single noise level curves. evident from these curves, that while the minima might not lie the modes -1 and +-1 for the single noise level curves, by averaging across multiple noise levels, the solution(s) to (18) approach the mode(s) of the source distribu- tion. As pointed is that this behavior is only observed for a good choice of the noise level range.? 7 Larger noise levels aid in moving between the modes and smaller noise levels help resolving the solution. Only using larger noise levels, for example, can smoothen out the landscape to the point that the modes are no longer discernible, thus resulting in erroncous estimates. We leverage the noise levels from pre-trained diffusion models that have learned the statistical structure sufficiently well, thus avoiding cumbersome tuning of the noise level range.
in $4, an important caveat
B.3 Gaussian Mixture Source Model
Finally, we consider scalar Gaussian mixture model (GMM) that can often model arbitrary com- plicated scalar distributions. We derive the score for 2-component GMM and then generalize the result to an arbitrary K component GMM.
Proposition 4. Consider a two-component scalar Gaussian mixture source,
P2 ps(3) = (s + (1s))) 02),
E 10.
(39)
"This only corresponds to choosing an appropriate lower bound on 1 - and should not be confused with tuning noise schedule.
17

---

Under the Gaussian smoothing model in (17),
(2HH11 (2H41-5464 ((()) N909 of
V Pog (())
(c((3((s))
10 ((())
+(1- (1- (11-)))2((((5)
(40)
1/02 of
where
(51(6)) ()) 7/: 7 -z))
(41)
(2()(s)) = (1) 220 o,)) (s) ( + +1- ())))))((()))).
(42)
(43)
Proof. We need to compute the score of the following distribution,
Ps. (s)) Pysto,,2 (7/5 + d,Z, &s(s))
(44)
Our goal, to this end, is to leverage Lemma to compute the score, thereby requiring expression for the conditional expectation (31). Since and 2N are independent, (44) can be written as a convolution between distributions,
Ps. (Bx(s)) (ps Pro: (s' + = (8)),
(45)
where
= :s .ss and Zo, == Jzzs.
(46)
In order to compute the conditional expectation, we first show that,
ps'+zo, (s
Zo. 31(s))sss =a')ps =a)
1
(2085((5) + 7o7m No {C((((())) a; jo + oî
(()
7:00 + to²
1033)(()) + n 05 Hog +o+
No0003 n 20 + of
+(1 (1- (1-)))2(3(0))))
(47)
where
( = ((8 ((0 o,). (=)(())) = (1) 7, 10 + oz) Z (3)(s)) = A + (1- (1 (1-N)cc(((())
We start by deriving the density functions for the random variables (46). Since the convolution between two Gaussians is Gaussian,
Py (a) = XN Tor fo + (1- N); 1003),
(48)
Pr.,, (w) (w; 0, of),
(49)
from which it follows, for example through the identities derived in [48, Sec IJ), that
p&, (5x()) = (=:(() 7:44, of + + + (1 (5x((=;))) (4)2 nu4, 2203 + af). We conclude the derivation of the conditional distribution by using Bayes' rule,
(50)
Ps Ps' /s/ (s' Zo. = 3s) a)ps (s' = a)
Pe (s' = als' + Zo. = ={(s)) = S Spr+an (s' Zo. = 5x(s)ss = a (s (s' = a') da
(51)
whereby simplifying the terms in the numerator further, we have
Ps ls + Zo. (s)]sss = a) = Pr., (Zo. = (s) - a) = (s))
(52)
Finally, by following the Gaussian product identities in [48, Sec 1]. between the distribution in (52) and (48) we reach the result in (47).
18

---

0.4
C17
0.3
0.2
0.1
00
Figure 5: Left: A GMM source with two equiprobable modes at -1 and 2. Two smaller modes are present at +2 and +4. Right: The minima of (16) lie (approximately) at the modes of the source distribution (black curve). Colored curves correspond (16) evaluated with T 1.
Now we can compute the expectation as,
(1075((5) + noin गo.7 + of + (1--)) +(1-A)
(x)) (cc(3((s))
E Ess''s' + Zo. = =3r(s)]
12038((5) + no 7t + of
]
(53)
Using Lemma we can express the score as,
(W//11 (s(s) 2/07 oī
V V.((s) log.s, (5)()) (+))
Ac((3((s))
12H22 (((s) 1o02 + of
]
+(1 (12-))2(3((3))
We can generalize the score to an arbitrary K K-component GMM by exploiting symmetries in (40).
Proposition 5. Consider K K-component Gaussian mixture source,
K
(s())
Σ N.
Σ =1
(54)
For the Gaussian smoothing model (17), the score at timestep is,
1
K
(7414-
(Tels - 5c(s) ?Ro? + of
Se((s) Pa. (5r(s)) =
cc(((((s))
(55)
Z(5r(s))
where
(((x(s)) N( 200 + +of),
(56)
=() = A..((((())
(57)
Similar to the BPSK case, since closed-form expression for the solution to (18) cannot be obtained, we simulate (16) for four-component GMM with two large equiprobable modes and two smaller modes. As shown in the colored curves on the right Figure 5, we again notice similar behavior where the modes are sharper at lower noise levels, with the sharpness decreasing at large noise levels. Hence, gradient- -aaed methods can leverage the randomly chosen larger noise levels to move between modes and use the smaller noise levels to resolve the estimate. On average, the minima of the loss approach the modes of the original GMM source, as shown by the black curve on the right in Figure 5.
19

---

C
Datasets
In this section, we particularize the signal models introduced in $2.1 RF signals. We first consider a single-carrier signal modulating, c.g., QPSK symbols, which can be expressed as,
(t) Cp. g(t pff.).
(58)
The second form corresponds multi-carrier signals, such as OFDM signals, where multiple symbols are modulated in parallel,
L-1
b(t)
=
Σ
Σ
Cp.e exp (j2mtt/L).
(59)
p=
E=
Both types of signals can be grouped in a single family of signals represented as,
L-1
((()=
Σ
Cp.t g(t - pT,, E) exp (j2xlt/1..
(60)
p=-ox
8=0
We use s(t) to represent the signal of interest (SOI) and as the vector representation for collection of consecutive samples thereof. Similarly, the interference is represented by b(t) or b.
C.1 Terminology
Before we describe the construction of our datasets, we will provide some terminology that we use in our description. This terminology meant supplement the details already provided in $5.1 in the main paper.
Symbol constellation: A digital constellation describes the underlying scheme for mapping bits to symbols. Typically used digital constellations (the mapping between bits and symbols) include binary phase shift keying (BPSK) and quadrature phase shift keying (QPSK). Specific BPSK and QPSK examples are depicted in Figure 6.
Pulse/Symbol shaping function: This corresponds g( in (60) and meant help limit the bandwidth of the signal. Each symbol Cp.1 is multiplied with pulse shaping function with different time offsets and a main lobe width T., helping smoothen the signal and remove high frequencies. In this work, we will make use of the root-raised cosine (RRC) pulse shaping for our single-carrier QPSK SOL On the other had, OFDM signals do not use an explicit pulse shaping function (as shown in (59)), i.e., (t.,) corresponds to rectangular function in (60).
20
10
(a) BPSK constellation.
(b) QPSK constellation.
(c) Effect of RRC filtering. (d) Effect of 45" phase.
Figure 6: Discrete constellations for BPSK and QPSK signal. Application of the RRC filter the time domain leads to interpolation between constellation point as shown in (c). Phase offsets in the waveform domain can be corrected by looking at the rotated constellation as (d).
Total number of subcarriers: This corresponds to L. in (60) and specifically to OFDM signals of the form (59). The subcarriers are complex exponentials of the form exp (j2mt/T. T.). J & (O...., with spacing 2T/ L in the (angular) frequency domain, and are orthogonal to each
20

---

other. Note that the weighted summation with complex exponentials can be interpreted as computing the inverse discrete Fourier transform (IDFT) of the pulse-shaped symbols, for which the inverse fast Fourier transform (IFFT) is used during implementation. Thus, in an interference- free setting, decoding the symbols from b(t) can be performed by computing the FFT of b(t).
Starting point: When decoding an RF waveform to extract the underlying symbols, we need them be fime-synchoonized specified frame of reference known the starting point, т. If synchronized correctly, this indicates the number of time samples to stream in before bit decoding can begin. If unsynchronized, this corresponds to shift in time that results in "non-natural" looking decoded constellations at all other time shifts except for the correct time-shift Here, decoding of u(t) can be performed either via an FFT for OFDM, or sampling after matched filtering for single-carriel signal.
C.2 Synthetic and Over-the-air RF Datasets
Below we describe the details relating to the RF signals used in our single-channel source separation (SCSS) experiments. To simulate representative types of RF signals, we generated single-carrier QPSK signals with RRC pulse shaping and OFDM signals using the NVIDIA Sionna library [45]. We also used over-the-air recordings corresponding to "CommSignal2" from the dataset in the RF challenge. We do not have explicit knowledge on CommSignal2 source model as is not provided in the challenge. Hence, are unable explicitly describe the pulse shaping function of such signal type leaving learning of such properties to our data-driven diffusion models instead.
RRC-QPSK. We use QPSK signals with root raised cosine (RRC) pulse shaping [18, Sec 1 11.3] as SOI across our experiments. These signals are used in a variety of different protocols ranging from low-power radios to cellular signals and even for satellite communication. For our experiments, we generated a dataset of 100,000 RRC-QPSK waveforms with the following parameters:
Modulation scheme: QPSK (2 bits per symbol)
Pulse shaping function: RRC, roll-off factor 0.5, spanning ~8 symbols
Default starting point: Ts =
QPSK-specific details:
Oversampling rate: 16 [samplex/symbol], using zero-pad gero-padding between symbols.
OFDM (QPSK/BPSK). For the OFDM signals (specifically, eyclic prefix OFDM), we generated waveforms with the following parameters (so as to simulate an 802. In WiFi waveform):
Modulation scheme: Either QPSK (2 bits per symbol) or BPSK (1 per symbol)
Total number of subcarriers (FFT size): 64
Symbol shaping: None as per (59) or rectangular window as per (60)
Other notes: Random time offset and random phase rotation for each time series realization
OFDM-specific details:
Number of non-zero subearriers: 56 (null DC subcarrier (E 0))
Length of cyclic prefix: 16 (25% of FFT size)
Number of non-zero symbols: 56 symbols per 80 samples (1792 symbols in 2,560 window)
Symbol length: 80 (64 subcarriers + cyclic prefix of length 16)
In our experiments, we choose to use OFDM signals as one class of interference signals. This means that, given new realization, we assume that the starting point of new OFDM symbol (i.e., m.) unknown. Further, we assume no access to knowledge about the parameters outlined above, such as the number of subcarriers, the FFT size and the length of cyclic prefix. Hence, we seek data-driven approach where characteristics of this signal should be learned through data.
21

---

Table 1: Hyperparameters used for training our diffusion models.
QPSK
OFDM (BPSK)
OFDM (QPSK)
CommSignal2
Number of residual blocks (Rt)
30
30
30
30
Dilation cycle length (m)
10
10
10
10
Residual channels (C)
64
128
256
128
Random shift phase rotation
X
V
v
v
Batch size
128
128
128
Learning rate
5e-4
5e-4
5e-4
1e-4
Early stopping iteration
360, 000
220, 000
340, 000
90, 000
CommSignal2. CommSignal2 is dataset comprising over-the-air recordings of type of communi- cation waveforms, provided in [44]. This dataset contains 150 frames each of length 43, 560 samples. For purposes of this work, we use the first 100 frames in creating our training set for training the diffusion model, and we extract windows of relevant lengths (2560 samples) from it, to create dataset of size 160, 000 samples. The last 50 frames are reserved for creating our test set. Similar to the OFDM case, we also perform random time offsets and phase rotations for each time series realization.
D Diffusion Models for RF Signals
We train diffusion models to learn the statistical structure inherent to digital RF signals. As detailed in Appendix C, these signals exhibit both continuous and discrete properties. To this end, we adopt an architecture based on an open-sourced implementation of the DiffWave [24] diffusion model, initially developed for speech synthesis.
Broadly speaking, as shown in [24, Figure 2, Appendix CJ, DiffWave employs I residual blocks with dilated convolutions, where the output of block i - serves as input block i E (0. (0...., R R-11 1}. The dilated convolutions assist in the learning of long range temporal and periodic structures. The dilations first start small and successively get larger, such that the dilation at block is given by 2 modmi m where m is the dilation cycle length. For example, if the dilation periodicity is m = 10, then in block i the dilation 512 and block 10 the dilation reset This allows the network to efficiently tradeoff between learning local and global temporal structures. All residual blocks use the same number of channels, C.
Modifications. Several changes have been made in order to facilitate training with RF signals. First, since we are dealing with complex-valuee valued continuous waveforms, we train on two channel signals where the real and imaginary components of the RF signals are concatenated the channel dimension. Second, while the open- soureed implementation uses an & loss for training. we train with an MSE (squared (2) loss to match the training objective in [21]. We monitor the validation MSE loss, and once the loss stops decreasing substantially, we choose to stop training early. Lastly, we had to increase the channel dimension to learn more complicated RF signals, e.g., OFDM (QPSK). We train all our models on 2 x NVIDIA 3090 GPUs. Additionally, during data loading, we perform random time shifts and phase rotations on the OFDM (BPSK), OFDM (QPSK) and CommSignal2 signals. Physically, these simulate transmission impairments in RF systems. The hyperparameters that we use for training each model are shown in Table 1.
Metrics. the image domain, metrics such as the Fréchet Inception Distance (FID) [49] can be used to assess the quality of generative models. However, such perceptual metrics are less relevant to the digital communications domain. In the RF domain, the fidelity is measured by the ability to extract the underlying transmitted bits. Hence, in the context of our synthetic datasets, we probe generated samples and compare the estimated received symbols with the underlying constellation. However, for real world signals such as CommSignal2, for which we do not know the system parameters, have no other means to assess the fidelity apart from looking at time-domain structure. This is another
Namely, multiplication the signal with complex exponential with random (imaginary) exponent. hhttps:// / thub. com/lmnt wwt./offff..cccoooooooooo/oooooo diffwave
22

---

Real companent
Real component
200
600
800
200
600
620
800
Imseiiany component
Imoginary component
WW 800
2033
200
2000
600
(231
Figure 7: Left: A ground truth RRC- RRC-QPSK time-domain waveform. Right: A sample generated by trained RRC-QPSK diffusion model. Evidently, the generated waveform resembles the true one.
motivation for studying RF source separation, as provides us with a framework to assess the quality of the learned statistical priors.
D.1 RRC-QPSK
We train a diffusion model on the RRC-QPSK dataset (see Appendix C.2). While a QPSK signal has four distinct constellation points, application of the RRC filter results in interpolation between points as shown in Figure 6c. Figure shows an example a ground truth RRC-QPSK waveform on the left. On the right show RRC-QPSK sample generated by diffusion model. While it is hard to judge whether has learned the underlying discrete structure, visually, the waveform seems to have characteristics similar to the ground truth.
-32
-10
+0.%%
Constelltioo of generated sample
After matched Shering
Constellation generated sample
After matihed fiteringg
Figure 8: Two samples generated from the diffusion model trained RRC-QPSK samples. Within each box, the image on the left is underlying constellation of the realization generated by the diffusion model. Notice that the RRC filtering results in interpolation between the QPSK constellation points. After applying MF, the effects of pulse shaping are reversed and the original QPSK constellation is recovered.
In Figure 8, we show that by probing generated samples for the underlying (interpolated) symbols, we indeed recover the constellation for an RRC-QPSK signal. Note that we can do such probing since we know the parameters and signal generation model for an RRC-QPSK signal. Furthermore. by performing MF and removing the effects of pulse shaping we are able recover the original QPSK constellation, thus demonstrating that the diffusion model has indeed successfully learned the discrete constellation along with the pulse shaping function from data samples.
D.2 OFDM (BPSK and QPSK)
We train two OFDM diffusson models, on the OFDM (BPSK) and the OFDM (QPSK) dataset respectively (see Appendix C.2). Figure 9, shows an example of an OFDM (BPSK) signal from our dataset. On the left we plot the real and imaginary components of the waveform. These components visually look like Gaussian noise and it is not evident that there is actually inherent structure to these signals. As mentioned in Appendix C.2, the OFDM signals can be demodulated using oracle knowledge about the FFT size and the cyclic prefix. Additionally, since signal undergoes random time shift and phase rotation, the underlying (rotated) BPSK constellation will only be visible when the time shift has been compensated for, i.e., the signal has been time-synchronized, as shown on the right in Figure 9,
23

---

Real component
1.4
500
1000
1500
2000
2500
Imaginary component
0.0
X
0.4
Figure 9: Left: A complex-valued OFDM (BPSK) source augmented with time shift of 8 samples and with a random phase rotation plotted in the time domain. In the time domain, the discrete structure is not discernibee and the time-domain waveforms visually look like Gaussian noise. Right: Extracting the underlying (rotated) symbols from the waveform on the left by demodulating the OFDM signal using oracle knowledge about the FFT size, cyclic prefix, and time shift.
X
Figure 10: Top: Probing six generated samples from the OFDM (BPSK) diffusion model recovers the underlying BPSK constellation after time synchronization. Bottom: Probing six generated samples from the OFDM (QPSK) diffusion model recovers the underlying BPSK constellation after time synchronization.
The top row of Figure 10 shows the recovered constellation for six generated OFDM (BPSK) symbols and the bottom row shows the same for six generated OFDM (QPSK) signals. Note that since there are some null symbols, i..,, unused subcarriers in the frequency domain, there is an additional "constellation point" the origin. In general the recovered constellations are clean, except for few symbols that lie slightly off the constellation. When converting back to bits the symbols are mapped the nearest constellation point.
D.3
CommSignal2
The CommSignal2 dataset was recorded over-the-air and hence, unlike the synthetic datasets, cannot probe without knowledge of the true system parameters and signal generation model, which not provided in [44]. These complications serve one of our motivations to develop data-driven source separation models. In the real-world, CommSignal2 could interfere with an SOI such as a QPSK signal for which the signal generation model is known. A learned prior for CommSignal2 could help mitigate such interference.
As shown in Figure 1. we observe that the generated CommSignal2 waveforms generally display similar global temporal structure to the ground truth waveforms. To further demonstrate that our diffusion model has truly learned the statistical structure of the signal, we carry out source separation experiments and demonstrate that leveraging this diffusion model outperforms conventional and leamiig-basel source separators.

---

reul
real
-2
500
1000
1500
2000
2500
500
1000
1500
2000
2500
imaag
imag
-2
0
500
1000
1500
2000
2500
500
1000
1500
2000
2500
Figure 11: Left: Ground truth CommSignal2 waveform from the dataset. CommSignal2 waveform from the learned diffusion model.
Right: generated
Sampling
.(4)
Matched Filter
s/n] Man (xT.)
Symbols
Demapper
Decoded bits
(1x(1)
18ymbols
Figure 12: Block diagram of the matched filtering demodulation pipeline used in our work. (First symbol at т, = 8, symbol period T, = 16, as detailed in Appendix C
E
Classical RF Interference Mitigation Techniques
E.1
Matched Filtering
Matched filtering (MF) is a signal processing technique commonly used in the demodulation of RF waveforms. exploits knowledge about the signal waveform to enhance the detection and recovery of the transmitted symbols/bits, and is optimal in the maximum-SNR sense for signals contaminated with additive Gaussian noise.
The basic principle involves filtering (or similarly viewed as correlating) the received sampled RF waveform with a known reference waveform called the "matched filter". The goal is maximize the ratio (SINR) at the filtered output, which consequently minimizes the error probability in the subsequent symbol detection when the noise is Gaussian. In the following segment, we develop the theory of the matched filter, with particular focus to our problem formulation.
Consider the baseband RRC-QPSK signal as described in Appendix C. Suppose, for the purposes of this exposition, that we adopt simple additive white Gaussian noise (AWGN) channel model, thereby representing received signal as
y(t) Σ co Ct -p.') + w(t)
(61)
(t) 5(t pTx)
-Tott-pTa) + w(t).
(62)
where Cp are the symbols from a QPSK constellation (see Figure 6b), * denotes the convolution operator, 5(-) the dirac delta fucnction, and w(l) ~ (0, is the additive noise in the
observed signal, statistically independent of all {cp). Of particular interest in this formulation is the transmit pulse shaping function (1), where we chose use the RRC function. (This due its favorable properties in practice; however, the optimal transmit filter choices and waveform designs
25

---

are outside the scope of this work, and we defer readers to related resources for more information on this topic [18. 36].)
At the receiver, we seek a receiver filter, Ira (1), such that the filtered and sampled output
g/i((t = grx(t) * (tx(t)
Eep pI')) gix(t) * w())
(63)
g)(())
u|nd = yau (onx))))) = E cp Ep - -p)Ts) + / |w(T) /w(())sa( 9rx - 7 r)dT
(64)
:=0[n]
= cn g() + Σ cn g(nn p)T,) + z
(65)
y
pofn
yo [m]
would maximize the output SINR. In other words, are looking to maximize
E V. [
E
[I 2 10(0))²
SINR
(66)
E 111. [1]
E [|cn| 2 E ppn (g(TT)) + AAaN J (GPPF
(where Gx is the Fourier transform of (t)( via an appropriate choice of g(t) and thereby, (t)(t). This can be done by finding an upper bound on the SINR that reaches equality for the appropriate filter choices. Ultimately, one such choice is Ir. (t) = -et)-temeed
termed as the matched
filler-that leads to a maximized SINR. the case of an RRC pulse shaping function (which is real and symmetric), the matched filter is also the same RRC function.
In this work, we refer matched filtering more broadly also include the detector/decoding step, as shown in Figure 12. As part of the demodulation pipeline, the filtered output is sampled (as in (65)), and then mapped to the closest symbol in predefined constellation (in the Euclidean distance sense). Finally, we can map these complex-valued symbols back to their corresponding bits to recover the underlying information. We use this as standard demooullation/detectioo pipeline for our RRC-QPSK SOI following the corresponding signal separation//nterfernnc mitigation step (where applicable).
Demodulation with matched filtering is optimal for waveforms in the presence of additive Gaussian noise. However, in our signal separation problem, we consider the presence of an additive interfer- ence, which is not necessarily Gaussian. Thus, exploiting the non-Gaussian characteristics of the interference would likely lead to enhanced decoding performance.
E.2 LMMSE Estimation
Recall that our observation model is
y S Rb, where we assume s and b are zero-mean and that they are statistically independent. The linear minimum mean square error (LMMSE) estimator is the estimator $ W IMMMEE, such that
W LMMSE
arg min E [[]s Wyll
(67)
WECTXT
In this case, the optimal linear transformation (in the sense of (67)) can be written
W LMMSE C.sy C C.s (C.s + x² Cts)
where Cay := E]sy"" corresponds to the cross-covariance between and y, Cay. C.s. Cso are the auto-covariance of y. and b respectively. The second equality is obtained by statistical independence, thereby C.y = C.s. Cyv = C.. + CCCbb.
In relevant literature, this may also be referred to as linear MMSE receivers [50, Sec 8.3.3]. Note that we aim to mitigate the effects of an additive interference channel. For the purposes of this work, we use LMMSE as one of the baselines Thereafter, we assess performance based on the squared error between with the ground truth s. To obtain the underlying bits, we perform standard matched filtering operation on the estimatorss S.
signal separation (interference mitigation) method.
26

---

To implement the LMMSE estimator described above, one simply requires the covariances C gy. Cas Ca.. Under the problem formulation, one could potentially obtain the sample covariance, whose precision scales with the number of samples used. In the case of the synthetic signals, we assume oracle knowledge of the covariance when computing this baseline, so as to eliminate confounding effects from poor estimation of these statistics. However, for CommSignal2, we assume that the signal is zero-mean (validated with the empirical mean), and compute the autocorrelation matrix for time windows of length 2560, across 10,000 sample realizations drawn randomly from the frames provided in the dataset. The empirical covariance estimate that we used will provided in the accompanying code/data repository.
Note that in this problem, the interference also has random time offsets, which has be accounted for when computing or estimating the covariance of our interference signals. The works in [43. 12] address the effects of such random time offsets in detail, and discuss the suboptimality of the above LMMSE in such a setting. We acknowledge that without an additional time synchronization step, the LMMSE estimator adopted herein may be modeling the interference as time series from stationary process.
We remark that the LMMSE estimator is optimal if the components were Gaussian. However, as digital communication signals contain some underlying discreteness and undergo unknown time- shifts, these signals are typically non- Gau-Gaasiinn (and often, even far from Gaussian). Hence, better performance can generally be obtained through nonlinear methods.
F BASIS for RF Source Separation
We the BASIS method [5] learning-based benchmark for SCSS in experiments. Similar to our method, BASIS leverages pre-trained generative models as statistical priors and does not rely on end-to-end (supervised) training with paired data. The manner in which these priors are used for separation, however differs:
1. Soft constraint vs. hard constraint: In the context of problem formulation, interested in decomposing a mixture y = s + &b into its constituent components. As described in $3.2 our proposed method recovers the components by extending MAP estimation rule with a hard constraint given by,
arg max Psl aES stt y=s+ab
(68)
where the resulting estimate of b is = (y - 8)/K. The result of imposing this constraint is MAP objective that leverages two priors but only requires optimization with a single
Algorithm 2 BASIS Separation (original, as proposed in [5])
1: function SEPARATION(Y: K. N. (o?) (00 (0) Specially tuned noise schedule {ot ]T
2: for 1 1, do
3:
(1) + 16 ())
bb
+ wb (1-1) Th - f(o))
b Learning rate Y)
4: for i < O,N
do
5:
6, 6 ~ ~N(. I)
6:
2. 20 To. (b)))
t). Ten (2("),t .t)
> Compute scores at noise level o
(6 + w!!) Th (Z. (o))) -Th (y 85-2be ( (b) kab (0)) l0t + /2m
7:
approx. score
50 (/ TT /tb/or) na (y- v6(() -xbb()) l08 + V2T
8:
9:
end for end for
(T) 6 ( 11: return S = (T)
(T)
12: end function
27

---

variable.
arg max Psly(sly) #ES y=s+ob
arg max Py/s (y)s()(s))sss)
(69a)
sES y=stab
arg min log (s) - log pb ((y - s)/k).
(69b)
sES
In contrast, BASIS strives to estimate the underlying components by sampling from the posterior. In particular, they do not enforce hard constraint during sampling and instead assume a likelihood of the form
BASIS Py|s.b (y|s, b) V (y;s xb,
(70)
which actually corresponds . [N00 72) such that,
alternate mixture model with auxilliary noise
y + xb + w.
(71)
Thus, the estimates & BASIS and b BASIS are obtained by sampling from the posterior,
BASIS Ps.bly
(s, bly) = Py/s.b (y|s, b) (b.))..
(72)
Notice that sampling from this posterior requires computing two separate estimates due to the soft constraint in (71).
2. Annealed Langevin dynamics vs. randomized levels of Gaussian smoothing: The aforementioned BASIS estimates satisfy y g + xb BASIS only when + 0 in the soft constraint in (71) enforced via the likelihood term. As shown in Algorithm 2, the idea behind BASIS is to leverage annealed Langevin dynamics sampling to sample from the posterior by slowly decreasing the value of y towards zero by tuning a specially chosen noise schedule. While generally a separate noise schedule is required per prior, the authors suggest sharing the same schedule across both priors. Apart from tuning the noise schedule, the learning rate schedule is another hyperparameter that must also be tuned. In contrast, our algorithm leverages the existing noise schedule from pre-trained diffusion models in a randomized fashion without any additional tuning.
3. Optimization via sampling vs. optimization via gradient descent: Starting with initializa- tions (0) and 4/ (0) for the SOI and interference respectively, BASIS refines the estimate such that (0) approximates a sample drawn from the distribution of the smoothened source Ps. Langevin dynamics leverages the approximate score of Ps. to update the estimate by sampling from higher density regions. A similar update is performed for the interference. The estimates constrained to satisfy (71) through the Gaussian likelihood.
In constrast, as described in Algorithm (see $3.2.2), we obtain an estimate ((t) of the unsmoothened SOI for each randomly drawn noise level. The hard constraint allows us to readily get an estimate for the interference as (y - (y-0(%)) then update the SOI estimate using gradient descent, with rule guided by (69b), using the gradients of the log densities with respect to 0 (which is functionally equivalent to the scaled scores). The ideal update gradient is,
As described in $3.2.2, we
Vel(O) = -E.z. [Var St.
(s (0)) + w
Eu.z. [Vanss. (b. (0,x))]
(73)
We first implement BASIS in its original form, with our best attempt at finding an appropriate learning rate. As shown in Figure 13, on QPSK (SOI) + OFDM (QPSK) mixture, this leads to worse performance than matched filtering- in (71). Note that we are limited in the annealing schedule by the set of possible noise levels, based on the training setup of our diffusion models. Thus, we use the diffusion model training noise schedule discretized into T = 50 timesteps (see $5.2) and additionally use N 100. We seek to use the largest set of noise levels possible, consistent with approaches in most Langevin sampling based works. We tuned the learning rate schedule to the best of our abilities and found that 7) faa? = (2c-8)of 8 / gave the best results. All curves use an analytical score model for the SOI.
due to poor matching of the observation model
To make an even more conservative comparison (w.rt. our proposed method), we replace the posterior using (72), which corresponds to the formulation in (71), with a hard constraint as in our setup. This
28

---

RRC-QPSK SOI + OFDM (QPSK) Interference
RRC-QPSK SOI OFDM (QPSK) Interference
10
MSEE
10-11
10"
Macio ROG
10
(R))
SIR
Figure 13: Comparing BASIS and Modified BASIS against conventional RF source separation methods. The vanilla BASIS algorithm performs worse than MF, while the Modified BASIS algorithm, which inspired by our usage of an a-posterior, is able to beat LMMSE. Still, our method outperforms all baselines.
Algorithm 3 BASIS Separation, using (69b)
function SEPARATION(Y, K, N, {o, 1T ((0))
Specially tuned noise schedule {or)tm
2: for / <- 1, do
3:
( +- ((t-1)
1/ + f(o?)
> Learning rate T)
4:
for i + 0,
do
5:
~ I),
2, 2, Té. (t(() .): rob (yy-(()) /R.
6:
Compute scores at noise level de
(()) + (t)) -1t (e//o)) + 1 (20/0r) 21/E
7:
from MAP objective
8: end for
9: end for
10: return 8 = (T) b (y-0(7)) //8
end function
Algorithm 4 Modified BASIS Separation by introducing o-posterior
1: function SEPARATION(Y, K, N, {o?)T= (0))
Specially tuned noise schedule {or {o.)T_
2: for t < 1, do
3:
( + (t-
17 + f(o?)
b Learning rate f)
4:
for i < 0, do
5:
6 ~ I),
2, 26 Tos (t)) .). roh (y-))()) /KC,
6:
> Compute scores at noise level d
7:
(1) (t)) m (2./or) + n (20/01) + 22m
from MAP objective with cx-posterior
end for 9: end for
8:
return S = (T) b (y-0 /
11: end function
method, as further described in Algorithm 3, leads to better performance, even outperforming LMMSE in some cases. We also experiment with N = 400 (for total of 20, 000 iterations), and do not observe significant gains. Empirically we found this method to be typically stuck in local optimum which is in line with our characterization earlier in $4.
Finally, we augment the BASIS algorithm with an a-posterior with a = w, as described in Algorithm 4. As shown in Figure 13, the axposttriirr augmented method obtains better performance than vanilla BASIS, but is generally unable to reach the performance of our method. We emphasize again that in contrast to method, the aforementioned BASIS-based methods follow prescribed annealing
29

---

schedule. that can require cumbersome tuning for optimal performance. The implementations of Algorithm and Algorithm both use learning rate schedule n f(o)) (2c-6)o]/of
G Complete Results
In this section we will provide the complete results of our study. We first present another baseline based on simulating the reverse diffusion process [21] as a denoiser, that complements the baselines already described in $5.2.
Reverse Diffusion. Given mixture y, we interpret the SOI as (scaled) additive noise on top of the interference b. Since the reverse diffusion process can be interpreted as an iterative denoiser (see $2.2), we run a small chain of reverse diffusion for 10 timesteps starting at a noise variance of 0.005 and ending with variance of 0.0001, which was chosen based on different trials. Similar ideas have been used in prior works involving inverse problems for images [31, 51J. We note that it is generally cumbersome to find the optimal reverse diffusion noise range and it might in fact be dependent on the specific mixture's SIR.
Extracting the underlying bits. Our primary metric for assessing the quality of the estimated SOI through the BER between the estimated bits and the truly encoded bits. The estimate of the SOI typically equals the true SOI with very small amount of noise due to the continuous nature of the optimization problem. Thus, we can expect the underlying constellation to contain some symbols concentrated around, but not exactly at the constellation points. This can be modeled as the true SOI corrupted with a small amount of Gaussian noise, for which MF is the optimal technique to decode the bits by mapping the symbols to the closest constellation point (see Section E.
The analytical SOI score. We use an RRC-QPSK signal as the SOI across all our source separation experiments. As detailed in Section B.2, the score for smoothened QPSK source can be analytically computed via Proposition 3, where is more amenable to model and compute (i.e., as i.i.d. symbols). Nevertheless, for the problem of separating the SOI from an interference source, we have to consider the components jointly in the time domain. Thus, we relate the time- domain representation to the symbols via s Ha, where H represents the RRC filter matrix and a is a vector of symbols. To compute this analytical score, we smooth the symbols via a Gaussian smoothing model at noise level corresponding to timestep t,
the symbol space
S; a; := H (maa + o,2s).
(74)
With this relation, we express the score of the smoothened source as,
V s, log Ps, (5:)
H V log Pa. (ā,)
(75)
H
1
à Σ a Oz
(76)
0
REAR
2 of
S + H Σ a a oe a: H
(77)
DEAK
where @ represents element-wise product, Φe is the softmax-like operator defined in (34) that is applied element- wise here, and H is the pseudo-inverse of H. Note that (76) is obtained by applying (33) to vector of i.i.d. smoothened QPSK symbols. In our implementation, the estimate of these smoothened symbols å, is obtained by reversing the RRC filter using H as in (77).
On the other hand, our RRC-QPSK diffusion model is trained directly on the time-domain waveform, and can be used directly to separate the SOI waveform from the mixture without conversion to the symbol domain. This once again sheds light on the practicality of using data-driven methods to circumvent otherwise computationally challenging statistical modeling technical problems.
Figures 14 and 15 show the complete source separation results for mixtures with OFDM and CommSignal2 as interference, respectively. Our model that uses an analytical SOI score for the SOI and a diffusion-bar based score for the interference generally performs the best and outperforms all baselines. Furthermore, we show that using a learned SOI score still outperforms all baselines in
Results.
30

---

RRC-QPSK SOI + OFDM (BPSK) Interference
RRC-QPSK SOI + OFDM (BPSK) Interference
307
Roow (iflowe bactomis A27
DACS Sypcaton
2MG Julnt +0
10-1
10
MSE
10-2
10
10
10"4
-20
-20
-10
SIR (DB)
SIR [BB)
RRC-QPSK SOI OFDM (QPSK) Interference
RRC-OPSK SOI OFDM (QPSK) Interference
LIDEIS Andiow S2
10
10
10-1
MBN
MSG
10-2
10-1
10
10-1
SIR (dB)
SIR (cB)
Figure 14: Comparing our method against various baselines on separating (top) RRC-QPSK- + OFDM (BPSK) mixtures and (bottom) RRC-QPSK + OFDM (QPSK) mixtures. Our model that uses an analytical SOI score outperforms all baselines in terms of BER. Reverse diffusion is competitive at low SIRs since the interference dominates the mixture in this regime and hence iterative denoising to separate the SOI is effective. The trained SOI diffusion models significantly outperform all baselines at high SIR. The reason the analytical SOI performs slightly worse is due to the approximations make in (77).
RRC-QPSK SOI- Commaignal2 Interference
RRC-QPSK SOI + CommSignal2 Interference
20'
1V
10
10P
ciocbbibb Voake
BBR
10
MSE
10-1
10-1
10
-25
-20
SIR 10
-25
-20
-25
-10 -5
SIR (dB]
Figure 15: Comparing our method against various baselines on separating RRC-QPS + CommSig- nal2 mixtures. Our method significantly outperforms all baselines in terms of BER. The large MSE at low SIRs suggests that there is background noise present in the CommSignal2 source, which is amplified at lower SIR (large k). We try to estimate the amount of noise and model as additive Gaussian noise. Accounting for this noise could presumably lead to a lower bound on the BER, shown by dotted black line on the left.
terms of BER, despite the slight degradation. The trained SOI score models consistently outperform all methods at high SIR in terms of BER since the SOI diffusion model was trained with RRC-QPSK samples as opposed to the approximations made in (77) in implementing the analytical score.
Our method also outperforms baselines in terms of MSE for OFDM mixtures as shown in Figure 14. The performance at low SIR in the context of CommSignal2 mixtures is not the same. As shown in Figure 15, the large MSE at low SIRs suggest that there is background noise present in the CommSignal2 sources that is amplified for large values of x, i.e., the interference is actually of the form b + for some background noise We validated that this was indeed the case, by visualizing
31

---

samples in both the time-domain and frequency domain using the RF challenge demo notebook '0 As shown in Figure 16, we noticed segments of lower magnitude the start and end, which we believe to be background noise that is not part of intended communication signal. During source separation, this presumably results in noisier estimate of the SOI in comparison to mixtures with no additional background noise. We estimated the SNR to be 16.9 dB by averaging across multiple samples. The dotted black curve on the left Figure 15 is a presumable lower bound on the BER by accounting for the magnitude of the background noise and modeling it additive white Gaussian noise.
Sample CommSignal2 Frame
Spectrogram of Sumple CommSignal2 Trame
0.0
Frequercy
-1.0
25.00
5000
7500
10030
12500
15000
17500
20000
Time Sumple
Time Asis
Time-domain and frequency-dome plots of single frame from the CommSignal2 dataset. Left: In the time-domain we observe segments of lower magnitude at the start and end, which we believe to be background noise. Right: In the frequency-domain we observe the bandlimited communication signal with spectrally flat features in the regions we identified as background noise.
Figure 16:
Nevertheless, across all experiments, demonstrate that our method a new state-of-the-art for applications such as interference mitigation where the interference can be learned from data recordings and the desired information can be decoded with knowledge of the SOI demodulation pipeline.
RRC-QPSK SOI OFDM (QPSK) Interference
RRC-QPSK SOI + OFDM (QPSK) Interference
10
10-1
10
DDK
MSE
10a
16
10
10-2
10a
10P
10P
RRC-QPSK SOI + OFDM (QPSK) Interference
RRC- RRRC-QPKKTT SOI + OFDM (QPSK) Interference
10-1
BER
MSS
107
10
10
10
22
Figure 17: Top: BER and MSE versus w/x² x2 for different SIR levels. Evidently, good choice of w, on average, across different noise levels is w x², Bottom: By looking individual SIRs we see that the minimum BER and MSE is achieved when increases with increasing x2 (decreasing SIR), visualized using a log scale on the x-axis.
10 notebook<Demmo. ipynb
32

---

RRC-QPSK SOI + OFDM (BPSK) Interference
RRC-QPSK SOI + OFDM (BPSK) Interference
10
The thald
C-Natig alou Salu 20
Uli Supartin Siae
DCR
10
MSE
10
10
10"4
SIR (JR)
STR (cR)
RRC-QPSK SOI OFDM (QPSK) Interference
RRC-QPSK SOI OFDM (QPSK) Interference
901 Maadi
10
Quis
10-1
BER
MSE
10
10-3
10-1
10
25
-00
-25
-00
15
-10
SIR (JB)
SIR (JB)
RRC-QPSK SOI CommSignaa2 Interference
RRC-QPSK SOI CommSiigal2 Interference
107
10
10'
10--
MSE
30
10
30
30
25
-20
-25
-20 -15
10
SUR
SIR (GB)
Figure 18: Comparing our method against supervised learning setup that trains a UNet on paired data samples with an &g loss. We show that we are able to achieve competitive BERs, and even outperform the supervised method at certain SIRs. Our method is particularly competitive in the challenging strong interference regime (low SIR), demonstrating the fidelity of our trained interference diffusson models. The gap larger for CommSignal2 mixtures as the supervised method presumably able to leverage knowledge of the joint statistics between the SOI and interference to effectively deal with the background noise in the interference signal.
Choice of w. We numerically verify that w x² is good choice of the a-posterior term, To this end, we first find a suitable order of magnitude for w. by varying w/rc² between 10-2 and 102 across different noise levels. As shown in the top row in Figure 17, on average, the minimum BER and MSE is achieved when w = x²2 We additionally validate that it is beneficial to adapt w as the SIR changes by varying w and studying the BER and MSE curves individual noise levels. As shown in the bottom row of Figure 17, we observe that it w should increase with x² to achieve good results.
Comparing with supervised methods. As motivated in $1, we are interested in leveraging indepen- dently trained priors in our source separation setup. Nevertheless, we compare against a supervised setup that learns to separate mixtures end-to-end by training a UNet on paired data. We train three supervised models based on the recent work in [43], using their open-sourced training co These models are trained with an E2 loss. As such, we should expect the MSE performance to be better than our unsupervised approach, which is indeed the case across all the mixtures as shown in Figure 18. However, we notice that our method, which uses an analytical SOI score, is able to perform similarly to the UNet and even better in terms of BER for some SIRs, especially in the OFDM interference setting. Furthermore, in the challenging low SIR (strong interference regime) our method performs
33

---

well, showing that our interference diffusion models were able to learn the underlying statistical structures. Thus, the mixture model changes in the future we can re-use our priors whereas the supervised approached could require entire change to the architecture and additional training. We can potentially drive the BER the CommSignal2 setting by modeling the background noise in these signals either through different mixture model or through additional priors might help in driving down the BER. This will be a focus future work.
Computation Time. The inference time is of now slightly longer than BASIS. To separate single mixture, our method that uses 20, 000 iterations takes 328 seconds on average, whereas BASIS takes 284 seconds for the same number of iterations. Meanwhile, the UNet only requires one forward pass through the model and can separate a mixture in less than one second. Our implementation was not optimized for computation time, but there may be strategies to speed up algorithm in practice, which will be a focus future work.
Code. The code for reproducing results can be found https://githi: com/tkj516/ str__aaee_ssrre_pprrrrreee alpha-rgs mata-rrg.tittul github.ii.... io.
and is also linked on our project webpage https://
34


Reviews:
[{"review_id": "bFE5M0NNH5", "reviewer": "Reviewer_jCQT", "summary": "This paper investigates a novel Bayesian approach for separating the superposition of two sources based on diffusion generative models. The problem is motivated by application to the spectrum of radio-frequency (RF) communication systems. Several experiments using real datasets on RF  mixtures demonstrate that the proposed method reduces the bit error rate, which is the stared measure of performance for this application.", "questions": "+ The paper lacks of theoretical results showing the limitation of the proposed method and the assumptions on the data distribution to be able to perform source separation. \n+ I strongly suggest the authors to further investigate other scenarios for which source separation is requested which could be of major interested for the ML community.", "limitations": "The limitation are not well discussed; In particular, it is not clear the underlaying assumptions necessary to  satisfy the source separation.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper are as follows: \n\n+ It investigates a well-known problem, i.e., source separation, by using modern tools, i.e.,  diffusion generative models. In particular, the use of diffusion generative models to model the prior in the Bayesian source-separation framework is novel and appealing. \n+ Several numerical results are provided by using real datasets. \n+ The paper is well-written and the contribution is clearly stated.", "weaknesses": "The weaknesses of this paper are as follows: \n\n+ The paper introduces the problem of multi-source separation. However, it focus on the case of two sources and the methodology specializes very much on the presence of only two sources. \n+ There is no theory or more specifically, there are no guarantees on performance of the proposed method to operate the underlaying sources. Notice that a lot have been done in this area and it would important to understand the limitations of the proposed method or at least to derive sufficient condition to ensure that the sources can be separated. \n+ The applications to future wireless communication systems are interested but very much limited. In particular, those are applications are not the main focus of the ML community. it would have been beneficial to show the proposed can be used to other type of data since they are many practical cases (e.g. sounds and speech) for which source separation is requested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to understand the technical contributions in our work. Below we shall address the raised questions and concerns.\n\n**Regarding the reviewer’s comments under “weaknesses”:**\n1. **Multi-source Separation:** To make the exposition clearer, we focus on the two-component case as mentioned in the beginning of our manuscript on line 3. We next explain how this readily defines a procedure for multi-source separation as well. Consider a three-component mixture $y = s + n + w$. This can be separated using our algorithm in two different ways — a) by treating the interference as $b = n + w$ in the first pass of our algorithm, we can obtain $\\hat{s}$, and then separating $\\hat{b} = y - \\hat{s}$ in a second pass we obtain $\\hat{n}$ and $\\hat{w}$; b) by using the three priors simultaneously to obtain estimates $\\hat{s}$, $\\hat{n}$ and using the constraint $y = s + n + w$. In terms of applications, a very important scenario in RF is the interference rejection scenario, where we are mostly interested in the recovery of our signal of interest only. In this setup, the interference could potentially describe the background/environment, which in practice could be a superposition of multiple devices. *Recovering the signals in a multi-source setting (particularly in a single-pass) is a subject of future work — and given its relevance in the context of such problems, we will make this clearer in our discussion and concluding remarks.*\n2. **Theory and performance guarantees:**  The conditions for perfect signal separation can be described for certain simple models. One such sufficient condition is as follows: If the two sources, $s$ and $b$,  are discrete and the super constellation (see Appendix C.1 in the supplementary)  is uniquely decodable, i.e., the mapping between the bit representation and symbol representation of the signals is unique, perfect recovery is possible. However, in this work we are focusing on general signal types and mixtures, where perfect separability is not necessarily guaranteed and performance bounds are not analytically tractable. A more involved characterization of our algorithm’s convergence to the modes beyond Section 4 can be found in Appendix B. *We agree that understanding the sufficient conditions and easy access to the theory in Appendix B are important. We will modify Section 4 of the final manuscript to include the sufficient condition above along with a condensed version of the  analysis from Appendix B.*\n3. **Relevance to ML community:** While we agree that image and audio separation are important areas of research, we found existing score-based source separation methods initially developed for these domains to underperform on RF data (e.g., BASIS). Moreover, the underlying discreteness of typical RF signals distributions present new challenges to the ML community that generally do not arise with image or audio data. Recently, there has been tremendous interest in using ML techniques for digital communications (please see some sample publications below) and hence we strongly believe that such new application areas are of interest to the broad ML community. There are a plethora of challenging problems in the wireless domain and we believe the ML community can help take significant steps towards better solutions. ML is without doubt a central technology to future wireless standards such as 6G, where the increasing demand of bandwidth combined with high reliability will require sophisticated interference rejection/source separation algorithmic solutions. We hope that this work can help shed additional light on new and rapidly developing areas of ML.\n\n**Regarding the questions and limitations:**\n1. **Sufficient conditions:** Thank you for pointing this out as this may have not been very clear from our manuscript. Please see the response above under “Theory and performance guarantees” for the sufficient condition. *We will include this condition in Section 4 of the final manuscript.*\n2. **Applications to other modalities:** The paper is intentionally focused on digital communication signals due to the fundamental differences of the signals' statistics, which is essentially a different realm that will (possibly, and perhaps most likely) require (at least) new building block for successfully functioning DNNs (where \"success\" is in terms of significant gains over classical model-based methods). *However, we agree with the reviewer that it would be interesting to extend our experiments to other modalities as well. We appreciate the suggestion and we will add these experiments to our future work.*\n\n**A few recent publications (apart from the closely related works already cited in the manuscript) that leverage ML for digital communications:**\n1. **Specific to single-channel source separation of RF/wireless signals with ML:**\n- M.Zhao, et al. Single-channel blind source separation of spatial aliasing signal based on stacked-LSTM.Sensors, 2021.\n- X. Hou and Y. Gao. Single-channel blind separation of co-frequency signals based on convolutional network. Digital Signal Processing, 2022 \n- H. Ma, et al. A novel end-to-end deep separation network based on attention mechanism for single channel blind separation in wireless communication. IET Signal Processing, 2023.\n2. **ML in wireless/RF communications for other problems:**\n- T.  O’Shea, et al. Over-the-air deep learning based radio signal classification.IEEE J.Sel.Topics Signal Process., 2018.\n- T. O’Shea and J. Hoydis. An introduction to deep learning for the physical layer.IEEE Transactions on Cognitive Communications and Networking, 2017.\n- Y. Eldar, et al. Machine Learning and Wireless Communications.Cambridge: Cambridge University Press, 2022.\n3. **NVIDIA Sionna Toolkit for Next Generation Communications Research (which we use in our experiments):**\n- Hoydis, Jakob, et al.\"Sionna: An open-source library for next-generation physical layer research.\" arXiv preprint arXiv:2203.11854 (2022).\n"}, {"review_id": "P1Ys3QI6uP", "reviewer": "Reviewer_J15P", "summary": "A new method for separating superimposed sources using diffusion-based generative models is proposed (alpha-RGS).\nThe method relies on separately trained statistical priors of independent sources and is guided by maximum a posteriori estimation with an α-posterior.\nExperimental results with RF mixtures demonstrate that the method results in a BER reduction of 95% over classical and existing learning-based methods.\n\n", "questions": "1. It would be interesting to see how the model performs in different channel conditions, with different types of noise other than additive white Gaussian noise (AWGN). This would give us a better understanding of the model's robustness to different noise conditions.\n\n2. The BER curve of the proposed model is noisy because it was generated using a small number of test examples. This noise will likely disappear if we sample more test examples, which will allow us to compute the exact improvement (in terms of dB) of the proposed model over the baseline.", "limitations": ".", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The authors propose a new method for source separation called α-RGS (α-posterior with Randomized Gaussian Smoothing).\nThe method uses the (approximate) score from pre-trained diffusion models to extend maximum a posteriori (MAP) estimation using generalized Bayes' theorem with an α-posterior.\nα-RGS outperforms classical signal processing and annealed Langevin-dynamics-based approaches for RF source separation.", "weaknesses": ".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and helpful questions. We shall now address the raised questions and concerns.\n\n**Regarding the questions:**\n1. **Statistics of noise:** Thank you for the question. We would first like to clarify that in a typical communication setup with a transmitter, channel and receiver, it is often assumed that the channel noise is AWGN. Mathematically, the output $y$ is related to the input $s$ as $y = s + w$ where $w$ is AWGN. In our source separation setup we consider an interference channel $y = s + b$ where $b$ is no longer constrained to be AWGN and thus departing the channel noise model. The generality of our approach is rooted in learning the underlying structures of the interference, which can often be far more complicated than AWGN.\n2. **BER curves:** *We will increase the size of the test set for generating the curves for the final version of the manuscript as much as possible.*"}, {"review_id": "7XJfidy3ds", "reviewer": "Reviewer_aSGC", "summary": "This paper focuses on the problem of single-channel source separation (SCSS) for RF signals with discrete nature. The authors propose to solve the SCSS problem using MAP and use pre-trained diffusion models to approximate the scores for both the source signal and inference signal. To avoid being stuck in a local minimum, the proposed method uses an $\\alpha$-posterior and optimizes across multiple noise levels. Experiments show that the proposed method significantly outperforms the baselines (both traditional and learned methods) in both MSE and BER.", "questions": "1) Could the authors describe more about the suitable conditions in line 205, page 6?\n2) In Figure 3, seems like when the SIR is large (around -5dB), the trained SOI model could even outperform the analytical one. Could the authors explain more about that?  \n", "limitations": "The authors have adequately addressed the limitations ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) This paper is well-written and it is easy to follow. Given that it is the first work that explores score-based models in the SCSS problem of RF signals and it achieves significant improvement, this work could definitely point out a new direction in this domain and potentially influence other researchers.\n2) The idea of using $\\alpha$-posterior and randomizing across multiple noise levels could be potentially useful for other score-based optimization problems not limited to the scope of SCSS in RF signals. \n", "weaknesses": "1) The proposed method requires to know the scale factor $\\kappa$ (or SIR) and the distribution of the interference signal. However, the scaling factor is not available in real-world scenarios. Also, the interference signals could come from different sources such as WiFi, and Bluetooth. As a result, it may not be possible to train a diffusion model that models all kinds of interference signals in the real world.  \n2) I think this paper lacks theoretical analysis about: 1) how the proposed algorithm 1 could lead to the local extremum of equation (14), and 2) how minimizing the approximated loss in (9) and (14) could lead to the solution of the original MAP (6b) \n3) It would be nice to have more ablation studies about: 1) optimizing across random noise levels vs fixed noise levels, 2) results with and without the zero mean noise in (12)\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and helpful questions. We shall now address the raised questions and concerns.\n\n**Regarding the reviewer’s comments under “weaknesses”:**\n1. **Knowledge about the scaling factor**: *This is an excellent point and we hope to address it in future work by introducing a novel extension to our algorithm that jointly optimizes over the range of possible scaling factors as well.* Many communication systems have power constraints and equalization capabilities, and with the endowment of such knowledge it is possible to estimate the signal to interference ratio (SIR) within reasonable margin. Equalization/normalization of the SOI’s power can be performed, e.g., by leveraging header information (metadata) and the SIR can be inferred from the mixture with the knowledge of the former. Thus, as a first step towards this goal, we assume knowledge of $\\kappa$ in this work. \n2. **Interference signals from different sources:**  *Developing a library of priors in a cost-effective manner will be a focus of our future work.* It is true that the wireless ecosystem is growing at a rapid pace. We envision a system of plug-and-play priors where the receiver is given access to an ML backbone with diffusion-based priors, ***each*** trained on a different signal type, e.g., Bluetooth or WiFi. During transmission, a simple detector might detect the presence of a Bluetooth interference signal and the receiver can plug in the learned prior for Bluetooth signals to recover the signal-of-interest. We believe that such technology is vital for future communication standards such as 6G and we will focus our future efforts on researching new ways to efficiently learn these priors. For details on our diffusion model training setup, please see Appendix D.\n3. **Theoretical results:** Due to space considerations, we included our theoretical results in the supplementary material. *However, we agree that it is beneficial to readers to include some of the developed theory in the manuscript itself. We will move the most important parts of this theory from Appendix B to Section 4 in the final manuscript.*   \n4. **Connection between formulations:** Section 4 introduces the asymptotic loss function that our algorithm minimizes in the limit of a large number of iterations. Appendix B builds on this, “dissects” equation (14) further, and analytically proves the “mode-seeking” nature of our algorithm for multivariate normal sources, digital RF signals (discrete signals) and Gaussian mixture sources. For each of these cases, we analytically solve for the individual terms in (14) and show that the extrema corresponds to the modes of the underlying source distribution. For the original MAP formulation (6b), we know that the solution(s) correspond to points of maximum probability, i.e., the modes. Hence, the solution of our algorithm, which optimizes (14), approaches the solution of (6b).\n5. **Ablation studies:** We decided to use all noise levels in our algorithm since diffusion models capture statistical structures at different resolutions based on the amount of noise added in the forward process. This is pictorially depicted in the middle and rightmost plots of Figure 1, where the distribution is smoothened and less peaky with increasing amounts of Gaussian smoothing.  We emphasize that the BASIS algorithm also uses all noise levels but the noise is gradually annealed over time via a ***fixed*** schedule. Each outer iteration of BASIS essentially tries to separate the components using a fixed noise level. As mentioned in the manuscript and in Appendix F, tuning this schedule is difficult, and more importantly still leads to underperformance in comparison to our method. As for the second ablation study without the subtractive noise, initial experiments demonstrated benefits with this additional term, but we will re-run the experiment for a few cases to be included in the supplementary material. We thank the reviewer for this suggestion.\n\n**Regarding the questions:**\n1. **Suitable conditions:** While it is analytically challenging to answer the necessary conditions under which this holds, we are able to provide sufficient conditions for separability of the sources under equation (14): If the two sources, $s$ and $b$,  are discrete and the super constellation (see Appendix C.1 in the supplementary)  is uniquely decodable, i.e., the mapping between the symbols in the super constellations of each source is unique, perfect recovery is possible. However, in this work we intentionally focus on general signal types and mixtures, where perfect separability is not necessarily guaranteed and performance bounds are not analytically tractable. Providing theoretical guarantees in more complicated scenarios is a topic of broad interest and we hope to address some of these problems in future work.\n2. **Trained model vs.analytical model:**  We attribute this degradation to numerical instabilities during the computation of the analytical score in the symbol domain (before pulse-shaping, i.e., $a$ where $s = Ha$). Computing the score of the pulse-shaped symbols ($s = Ha$) is extremely challenging. For more details on the digital communication pipeline please refer to Appendix C. Appendix G ((G.1) – (G.4)), details the calculation of the analytical score for the QPSK SOI in our experiments. As shown in (G.4), we make an approximation via the pseudo-inverse of $H$. At low interference levels (high SIR), this approximation can lead to small errors in symbol recovery, due to the negligible additive noise. On the other hand, the trained model directly learns the score of $s = Ha$ and thus circumvents such approximation errors.\n"}, {"review_id": "IqXB7h8hBC", "reviewer": "Reviewer_D4rU", "summary": "This paper proposed a new Bayesian method to separate a mixture of two signals $\\mathbf{s}$ and $\\mathbf{b}$ from the mixture signal $\\mathbf{y}=\\mathbf{s}+\\kappa \\mathbf{b}$ where $\\kappa \\in \\mathbb{R}_+$ is known. The new method leverages the score from pre-trained diffusion models to extend MAP estimation using generalized Bayes' theorem with an $\\alpha$-posterior across different levels of Gaussian smoothing. Experiments on RF sources demonstrate superior separation performance, with gains of up to $0.95$ in terms of both BER and MSE over classical and existing score-based source separation methods such as BASIS by Jayaram and Thickstun (ICML 2020). ", "questions": "+ In your plots in Fig. 3, $\\alpha$-RGS outperforms classical methods such that MF only or LMMSE +MF. Why can this fact hold? \n+ Why can your method outperform BASIS for equiprobable multimodal distributions?", "limitations": "This is a theoretical work. The authors already finished the checklist as well as stated the limitations of theorems and results.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Using deep learning for source separation problems has appeared in much research literature. The key contribution of this paper is to provide a new separation method that works well for a superimposition of two discrete sources which produces a joint distribution with multiple equiprobable modes.  \n\n\nThis work can be considered a novel combination of well-known techniques. ", "weaknesses": "The main weak point of this work is a lack of theoretical analysis or intuitions behind the superior separation performance of the proposed method.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments regarding our technical contributions. Below we address the raised questions and concerns.\n\n**Regarding the reviewer’s comments under “weaknesses”:**\n1. **Theory:** Due to space considerations, we included our theoretical results in the supplementary material. *However, we agree that it is beneficial to readers if we include the main parts  of the developed theory in the manuscript itself. We will move the most important parts of the theory from Appendix B to Section 4 in the final manuscript.* Currently, Section 4 introduces the asymptotic loss function that our algorithm minimizes in the limit of a large number of iterations. Appendix B builds on this, “dissects” equation (14) further, and analytically proves the “mode-seeking” nature of our algorithm for multivariate normal sources, digital RF signals (discrete signals) and Gaussian mixture sources. For each case, we analytically solve for the individual terms in (14) and show that the extrema corresponds to the modes of the underlying source distribution. For the original MAP formulation (6b), we know that the solution(s) correspond to points of maximum probability, i.e., the modes. Hence, the solution of our algorithm, which optimizes (14), approaches the solution of (6b).\n\n**Regarding the questions:**\n1. **MF and LMMSE:** We recall that MF is the optimal separation solution only when the interference $b$ is Gaussian (see Appendix E.1). Thus, when $b$ is not Gaussian, MF is simply suboptimal and our algorithm outperforms it. The LMMSE estimator is a ***linear*** estimator that minimizes the MSE between the estimated  and true signal. The LMMSE solution is only optimal when both sources are Gaussian. On the other hand, the learned diffusion-based priors model ***non-linearities*** (i.e., it generally extends beyond linear operators) in the data and hence can leverage these during separation. For more details on MF and LMMSE solutions please refer to Appendix E.\n2. **BASIS:** Please note that Appendix F of the supplementary contains a detailed explanation with the reasons for our model’s superior performance over BASIS. Having carefully reexamined the content of this appendix, we believe that it contains all the necessary clarifications regarding this point. Note that we also provide details about the implementation of BASIS in our experiments. In addition to the above, *we will incorporate some of this intuition into the final version of the manuscript.*\n"}, {"review_id": "00JnsTpOon", "reviewer": "Reviewer_AhTo", "summary": "1.\tIn this paper authors proposed a method for separating superimposed sources using diffusion based generative models. The proposed model derived a new objective function based on maximum a posterior and alpha posterior.\n2.\tThe application of the proposed work is clearly mentioned by considering the existing components in the system, The data driven score based single-channel source separation technique are proposed for signal septation which perform superior compared to the conventional methods.\n3.\tThe contribution of the proposed work are mainly.\na.\tBayesian method for single-channel source separation is used.\nb.\tScore Distillation Sampling (SDS) has got good results of local extrema loss.\nc.\tThe proposed method performs well compared to the signal processing and annealed Langevin-dynamics-based approaches for RF source separation method and the results are encouraging.\n4.\tThe results presented in this paper are highly encouraging. \n", "questions": "1.\tJustify randomizing across multiple noise levels, the local extrema of loss.\n2.\tWhat is the value of k is chosen; relative scaling coefficient between the two signals\n3.\tExplain the experimental set up used to derive results presented in Figure 1.\n4.\tIncrease the resolution of the Figure 2 and explain it in detail.\n5.  A detailed explanation of the diffusion model is required. papers related to diffusion model, and it can be applied to the proposed method need to be elaborated more.\n6.\tWhy the probabilities are converted to negative in the equation 6b justify it.\n7.\tA Comparison is required how the computational complexity of the proposed work is reduced compared to the conventional methods.\n8.\tTrain and test ratio used for the implementation is 90:10 is there any specific reason for it.\n", "limitations": "The author need to address the questions and revise the paper and submit it back.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Paper is very well organized.", "weaknesses": "The experimental set set up should be clearly explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and comments. We will address the raised questions and concerns here.\n\n**Regarding the reviewer’s comments under “weaknesses”:** \nDue to space considerations, we were not able to include all details in the main manuscript, but we have included them in the supplementary material. Appendix C provides details about the datasets and RF terminology. Appendix D provides architecture, implementation and training details about the diffusion models we used in our experiments. Appendix E provides insight into the classical baselines of matched filtering and LMMSE. Appendix F provides theoretical and implementation details about the BASIS algorithm. Appendix G presents our source separation experimental setup with a detailed overview of the results. Our intention with these multiple appendices is to help readers, and provide a paper that is as self-contained as possible,\n\n**Regarding the questions:**\n1. **Multiple noise levels:** The reason for using multiple noise noise levels in a randomized fashion is related to the dynamics of the diffusion process. Adding noise at different levels allows the optimization algorithm to explore the landscape of the distribution at different resolutions. At low noise levels, the distribution might remain peaky and an iterative gradient-based algorithm could get stuck at suboptimal local minima (the valley between two peaks). On the other hand, at larger noise levels, the distribution naturally appears more smoothened and allows for the optimization routine to escape from local minima. Hence, using multiple noise levels allows us to trade off between exploration of the optimization landscape on one hand, and resolving the solution with a sufficiently high-resolution landscape on the other hand. Please note that this is explained in Section 3.2.1 and 3.2.2 (line 164) and Section 4 of the paper. *In the final version of the manuscript, we will better underscore these explanations.*\n2. **Choice of $\\kappa$:** In our source separation experiments we vary the relative scaling of the two components. As mentioned on line 301 and as shown in Figure 3, SIR ranges from $-24$ dB to $-3$ dB. To map this SIR range back to the range of $\\kappa$s, the mapping on line 295 can be used. For example, given unit power sources $s$ and $b$, to construct a mixture with SIR -24 dB, the scaling factor required is $\\kappa =15.85$. \n3. **Experimental setup for Figure 1:** As stated in line 207, we consider two independent sources — $p_{\\textsf{s}}(s)=0.5$ where $s \\in \\\\{-1, +1\\\\}$ and $p_{\\textsf{b}}(b)=0.25$ where $b \\in \\\\{-2/\\sqrt{20}, +2/\\sqrt{20}, -6/\\sqrt{20}, +6/\\sqrt{20}\\\\}$. The joint distribution $p_{\\textsf{s}, \\textsf{b}}(s, b)$ has 8 equiprobable modes as shown in the leftmost plot of Figure 1. We randomly choose $s$ and $b$ to construct a mixture $y = s + \\kappa b$ where $\\kappa=15.85$, which corresponds to an SIR of $-24$ dB. If one plotted the line $y = s + 15.85 b$ in the $s$-$b$ plane it would appear as shown in Figure 1, with an intersection at the mode corresponding to the component values $(s, b) = (+1, -2/\\sqrt{20})$. As described in Section 4, our algorithm asymptotically minimizes (14) in the limit of a large number of iterations. The central plot of Figure 1, plots (14) with $\\omega=1$. Notice that the loss function is quite peaky and, especially, the incorrect mode $s=-1$ is pronounced even when the Gaussian smoothing is quite large. In the rightmost plot we demonstrate that with a suitable choice of $\\omega=\\kappa^2$, the optimization landscape can be made more amenable to gradient descent techniques, thus allowing convergence to the desired mode $s=+1$. *With the additional page provided in the final manuscript, we will make this explanation more clear.*\n4. **Enlarging Figure 2**: *We will enlarge Figure 2 in the final manuscript as much as possible.* Due to space considerations, we have deferred the relevant terminology required to understand Figure 2 in greater detail to Appendix C in the supplementary.\n5. **Literature review of diffusion models:** Thank you for this suggestion. *We will provide a more thorough review of diffusion models in the final manuscript/appendix.*\n6. **Negative probabilities:** From (6a) to (6b) we are interested in converting a maximization problem to a minimization problem. Since the solution to a maximization problem is equivalent to minimizing the negated objective, the probabilities now have a negative sign.\n7. **Computational complexity:** The primary objective in this study is to introduce a novel algorithm for source separation based on independently trained priors. While optimizing for the computation time is out of the scope of this paper, we make efforts to keep the implementation as efficient as possible (please take a look at the attached code) and compare the runtime of our algorithm to BASIS in Appendix G of the supplementary. *Nonetheless, we appreciate the suggestion of the reviewer and will look into optimizing the implementation for future work on the topic.  We will also move the former runtime comparisons to the results section in the final manuscript.*\n8. **Train-test split:** The 90-10 train-test split is a standard choice for splitting the RF Challenge dataset samples (as is done in the example notebooks from the challenge’s GitHub page). We decided to use the same split in the example notebooks for our data as well. While optimizing the training and test ratio could potentially yield improved outcomes, our primary objective in this study is to introduce the novel $alpha$-RGS algorithm. As a result, our focus remains on showcasing the algorithm's potential rather than extensively fine-tuning all implementation details for optimal performance.\n"}]

Author rebuttal:
We thank the reviewers for taking the time to review our work proposing a novel algorithm that leverages diffusion-based priors and randomized levels of Gaussian smoothing with applications source-separation in the RF domain, a relatively new domain area to the ML community.  We appreciate all the comments and questions that have been raised in the reviews, and by this global response we hope to respond to the reviewers collectively so as to address a few concerns.

Regarding the relevance of source separation in the context of wireless signals, we would like to reiterate that the source separation problem within the RF domain comes with a different set of technical challenges as well as academically interesting questions. We believe that such challenges are not only of interest to the ML community, but moreover, ML researchers can help shape novel algorithms for AI-enhanced next-generation communication technology (and indeed this is the vision for 6G).

As for the theoretic aspects of our work, due to space constraints, the development and understanding of our proposed algorithm from first principles was deferred to Appendix B.  Taking into account the reviewers’ comments, and in order to enhance readability, we will move some of the important theoretical results from Appendix B to the manuscript. We also appreciate the immense interest in a deeper understanding of the theory, and while we are able to provide sufficient conditions for separability in simple cases, there are indeed parts of the problem which might  be analytically intractable—e.g.conditions for perfect separability or the lower bound on the performance for general classes of signals—which are intriguing theoretic investigations which we will pursue in future work.

We understand that a detailed explanation of the experimental setup, digital communications terminology, details about the baselines and a detailed overview of the results are beneficial to readers. However, space constraints preclude us from including extensive details in the manuscript beyond what has already been provided, and are therefore included in Appendices C - G. With the additional space provided in the final manuscript, we will add a paragraph that describes the paper and appendix organization to help readers navigate the work efficiently. We will also move additional important details regarding the experimental setup to Section 5 in the final manuscript.


Meta-review: This paper proposes a new method for source separation based on generative models. The motivation comes from RF systems and experimental results with RF mixtures are provided.

The idea of using generative models for the prior in the Bayesian source-separation framework is interesting and novel, the empirical results are convincing, and the paper is well written. The authors have provided a thoughtful rebuttal which has addressed a number of concerns expressed by the reviewers, who have now reached a consensus towards accepting the paper. Based on my own reading of the reviews, rebuttal and paper, I agree with this view and I am happy to recommend acceptance.
Decision: Accept (poster)