SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Contribution and Distinction from Existing Works. Our method is in the effort of i) using score-based/diffusion-based methods beyond sampling, and to solve inverse problems, and ii) developing source separation strategies with a focus on sources with underlying discrete properties, which may make it differ from existing works where the usual goal is to obtain perceptually close solutions, which is not a good criterion in the RF domain.
Scalable Approach to Source Separation with Pre-Trained Models. An attractive property of our method is that by using pre-trained models as independent priors, we minimize the need for expensive compute resources to retrain models to tackle different scenarios/different mixture settings.
Trade-off in Computation Time. While the training time is reduced, the inference computation time is as of now longer than other methods (e.g., end-to-end training with mixtures as inputs and SOIs as desired outputs), thus making it unsuitable for real-time operation. As we discuss about its potential use for applications, speeding up the algorithmic aspects of the operation is essential in furthering this method towards practical applications.
Ethical Considerations of our Work. The intersection between generative AI and RF signals is a nascent field and hence, their ethical implications have been less explored. Nevertheless, we believe that this is an area that requires special attention to address issues such as malicious parties learning and spoofing, among others. While these are not the focus and the intent of our work, we believe that works in security in this space are actively exploring such problems.
Privacy Considerations with Generative Models. Where privacy and sensitive information is concerned, our work solely focuses on synthetic and public datasets. No sensitive transmissions have been captured, and therefore learned, by our generative models.
A.2 Regarding Applications to RF Systems
Novelty, Relevance and Timeliness. To the best of our knowledge, this is the first work that uses score-based methods for physical layer transmission in digital communication systems. The interference mitigation problem has significant relevance to enabling reliable operations in next-generation wireless systems. With the growth of heterogeneous wireless networks, there is overcrowding and growing scarcity of wireless spectrum. This work explores interference mitigation solutions with the goal of enabling efficient sharing of the finite resources to allow growth of the wireless ecosystem in a sustainable way.
Proof-of-Concept Exploration. Our work demonstrates via a proof-of-concept what might be possible for non-orthogonal access<sup>5</sup> to explore opportunities for new generation of transmission strategies and communication protocols; particularly, the emphasis on the low SIR regime is in line with motivations in lower-power device operations.
Considerations arising from Demonstration via Computational Simulations. All our experiments are done via computational simulation and thus, we did not operate in RF bands that we do not have rights to access. Our synthetic simulations may differ from practice where unmodeled physical phenomenon may occur, thus necessitating natural extensions of our work to deal with such impediments.
5That is, neighboring communicating devices that operate of the same, shared physical resources (e.g., time and frequency).
B Additional Characterizations of  $\alpha$ -RGS
We now "dissect" (15) in  $\S 4$ , and focus on a single term,
$$
\mathcal {L} _ {\mathbf {s}} (\boldsymbol {\theta}) := - \mathbb {E} _ {\mathbf {t}, \mathbf {z} _ {s}} \left[ \log p _ {\tilde {\mathbf {s}} _ {t}} \left(\tilde {\mathbf {s}} _ {t} (\boldsymbol {\theta})\right) \right], \tag {16}
$$
where we recall the definition of the Gaussian smoothing model from §3.2.1,
$$
\tilde {\mathbf {s}} _ {t} (\boldsymbol {\theta}) = \sqrt {\alpha_ {t}} \bar {\boldsymbol {\theta}} + \sqrt {1 - \alpha_ {t}} \mathbf {z} _ {s}, \quad \mathbf {z} _ {s} \sim \mathcal {N} (0, \mathbf {I} _ {d}). \tag {17}
$$
As detailed in §3.2.1, when estimating a discrete source  $\mathbf{s}$ ,  $\bar{\theta} = \theta + \varepsilon_s$  for  $\varepsilon \sim \mathcal{N}(0, \sigma_s^2\mathbf{I})$ ,  $\sigma_s \to 0$ , is a continuous surrogate of the discrete estimate  $\theta$ , useful for optimization via gradient descent. The above loss only depends on the prior of the source, and is in particular independent of the data. Therefore, this term serves (and can be viewed) as a regularizer in our inference optimization problem. Although analysis of (16) is (strictly speaking) not useful in order to make statements about (15), it is nevertheless informative and insightful to show that the local extrema of (16) approach the modes of the underlying source distribution  $P_{\mathrm{s}}$ , by solving for the stationary points,
$$
\mathbb {E} _ {\mathrm {t}, \mathbf {z} _ {s}} \left[ \nabla_ {\boldsymbol {\theta}} \log p _ {\bar {\mathbf {s}} _ {t}} \left(\gamma_ {t} \boldsymbol {\theta} + \sigma_ {t} \mathbf {z} _ {s}\right) \right] = 0 \tag {18}
$$
where we have used Leibniz rule for differentiation under the integral and where we have defined the shorthand notation,
$$
\gamma_ {t} := \sqrt {\alpha_ {t}} \quad \text {a n d} \quad \sigma_ {t} := \sqrt {1 - \alpha_ {t}}. \tag {19}
$$
Through our examples we will demonstrate the mode-approaching nature of the solutions to (18) thus establishing another interpretation of the asymptotic behavior of our method—two loss terms whose stationary points are modes of the corresponding source distributions which work together in unison to satisfy the constraints imposed by the observed mixture. We thereby end up, with high probability, in the correct mass-points of the underlying distributions.
B.1 Multivariate Normal Sources
We first start with the analysis of a multivariate normal source and show that the exact mode is obtained as the local extremum of (16).
Proposition 2. Let  $\mathbf{s} \in \mathbb{R}^d$  be a multivariate normal source with mean  $\mu_{\mathrm{s}}$  and covariance matrix  $\Sigma_{\mathrm{s}}$ . For the Gaussian smoothing model in (17), the score at timestep  $t$  is
$$
\nabla_ {(\tilde {\mathbf {s}} _ {t} (\mathbf {s}))} \log p _ {\tilde {\mathbf {s}} _ {t}} (\tilde {\mathbf {s}} _ {t} (\mathbf {s})) = - \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1} \left(\gamma_ {t} \mathbf {s} + \sigma_ {t} \mathbf {z} _ {t} - \boldsymbol {\mu} _ {\tilde {\mathbf {s}} _ {t}}\right), \tag {20}
$$
$$
\boldsymbol {\mu} _ {\bar {\mathbf {s}} _ {t}} := \gamma_ {t} \boldsymbol {\mu} _ {\mathbf {s}} \quad a n d \quad \boldsymbol {\Sigma} _ {\bar {\mathbf {s}} _ {t}} := \gamma_ {t} ^ {2} \boldsymbol {\Sigma} _ {\mathbf {s}} + \sigma_ {t} ^ {2} \mathbf {I}.
$$
Then, the minimizer  $\theta^{*} = \arg \min_{\pmb{\theta}}\mathcal{L}_{\mathbf{s}}(\pmb {\theta})$  is equal to  $\mu_{s}$ , i.e., the mode of the source distribution.
Proof. Since  $\mathbf{s}$  is a continuous source, we perform optimization over a continuous space with respect to  $\pmb{\theta} \in \mathbb{R}^d$ . Notice that,
$$
\nabla_ {\boldsymbol {\theta}} \log p _ {\tilde {\mathbf {s}} _ {t}} \left(\gamma_ {t} \boldsymbol {\theta} + \sigma_ {t} \mathbf {z} _ {s}\right) = \gamma_ {t} \nabla_ {\left(\tilde {\mathbf {s}} _ {t} (\boldsymbol {\theta})\right)} \log p _ {\tilde {\mathbf {s}} _ {t}} \left(\tilde {\mathbf {s}} _ {t} (\boldsymbol {\theta})\right) \tag {21a}
$$
$$
= \gamma_ {t} \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1} \left(\gamma_ {t} \boldsymbol {\theta} + \sigma_ {t} \mathbf {z} _ {s} - \boldsymbol {\mu} _ {\tilde {\mathbf {s}} _ {t}}\right) \tag {21b}
$$
Substituting (21b) in (18), we see that the minimizer must satisfy
$$
\mathbb {E} _ {\mathbf {t}} \left[ \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1} \left(\gamma_ {t} \boldsymbol {\theta} - \boldsymbol {\mu} _ {\tilde {\mathbf {s}} _ {t}}\right) \right] = 0, \tag {22}
$$
where we have used the fact that  $\mathbf{z}_s$  is a zero mean Gaussian realization. Thus, the minimizer is
$$
\boldsymbol {\theta} ^ {*} = \left(\sum_ {t = 1} ^ {T} \gamma_ {t} \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1}\right) ^ {- 1} \left(\sum_ {t = 1} ^ {T} \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1} \boldsymbol {\mu} _ {\tilde {\mathbf {s}} _ {t}}\right), \tag {23}
$$
which can be simplified further as
$$
\boldsymbol {\theta} ^ {*} = \left(\sum_ {t = 1} ^ {T} \gamma_ {t} \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1}\right) ^ {- 1} \left(\sum_ {t = 1} ^ {T} \gamma_ {t} \boldsymbol {\Sigma} _ {\tilde {\mathbf {s}} _ {t}} ^ {- 1}\right) \boldsymbol {\mu} _ {\mathbf {s}} = \boldsymbol {\mu} _ {\mathbf {s}}, \tag {24}
$$
where we have used (2) to substitute  $\pmb{\mu}_{\bar{\mathbf{s}}_t} = \gamma_t\pmb{\mu}_{\mathbf{s}}$
Thus, we have established that the local extremum of the (16) corresponds to the mode in the multivariate Gaussian case.
In this continuous setting, a surrogate distribution is not needed and  $\bar{\theta} = \theta$  in (17)
B.2 Tweedie's Formula and Digital RF Sources
We next analyze (16) in the context of RF source separation by studying sources defined by a digital constellation with symbols from the finite set  $\mathcal{A}$ . Before proceeding, we first state Tweedie's formula [47], a tool that will aid us in computing the scores of arbitrary distributions. For completeness of the exposition, we prove it for the case of scalar random variables below.
Lemma 1. (Tweedie's formula) Let  $\mathsf{a}$  be a random variable with an arbitrary distribution and let  $z_{\sigma}$  be a normal random variable with mean  $\mu$  and variance  $\sigma^2$  independent of  $\mathsf{a}$ . The score of the random variable  $x := \mathsf{a} + z_{\sigma}$  is
$$
\nabla_ {x} \log p _ {\times} (x) = \frac {\mu - x}{\sigma^ {2}} + \frac {1}{\sigma^ {2}} \mathbb {E} _ {a | x} [ a | x = a + z _ {\sigma} ]. \tag {25}
$$
Proof. Since  $\times$  is a sum of two independent random variables, it follows a convolution distribution given by,
$$
p _ {\mathrm {x}} (x) = \left(p _ {\mathrm {a}} * p _ {\mathrm {z} _ {\sigma}}\right) (x). \tag {26}
$$
$$
\begin{array}{l} \nabla_ {x} \log \left(p _ {\mathbf {a}} * p _ {\mathbf {z} _ {\sigma}}\right) (x) = \frac {1}{\left(p _ {\mathbf {a}} * p _ {\mathbf {z} _ {\sigma}}\right) (x)} \left(p _ {\mathbf {a}} * \nabla_ {x} p _ {\mathbf {z} _ {\sigma}}\right) (x) (27) \\ = \frac {1}{\left(p _ {\mathbf {a}} * p _ {\mathbf {z} _ {\sigma}}\right) (x)} \int p _ {\mathbf {a}} (t) \nabla_ {x} p _ {\mathbf {z} _ {\sigma}} (x - t) d t (28) \\ = \frac {1}{\left(p _ {\mathrm {a}} * p _ {z _ {\sigma}}\right) (x)} \int p _ {\mathrm {a}} (t) p _ {z _ {\sigma}} (x - t) \frac {(\mu - x + t)}{\sigma^ {2}} \mathrm {d} t (29) \\ = \frac {\mu - x}{\sigma^ {2}} + \frac {1}{\sigma^ {2}} \int t p _ {\mathrm {a} | \mathrm {a} + \mathrm {z} _ {\sigma}} (t | x) \mathrm {d} t (30) \\ = \frac {\mu - x}{\sigma^ {2}} + \frac {1}{\sigma^ {2}} \mathbb {E} [ a | x = a + z _ {\sigma} ]. (31) \\ \end{array}
$$
where (29) follows from,
$$
\nabla_ {x} p _ {\mathbf {z} _ {\sigma}} (x - t) = - p _ {\mathbf {z} _ {\sigma}} (x - t) \frac {(\mu - x + t)}{\sigma^ {2}}. \tag {32}
$$
Proposition 3. Let  $s$  be a scalar random variable representing a symbol drawn uniformly from the finite constellation set  $\mathcal{A} \subset \mathbb{C}$ . For the Gaussian smoothing model in (17) where, at timestep  $t$ ,
$$
\tilde {s} _ {t} (s) = \gamma_ {t} s + \sigma_ {t} z _ {t},
$$
the score is
$$
\nabla_ {(\tilde {s} _ {t} (s))} \log p _ {\tilde {s} _ {t}} (\tilde {s} _ {t} (s)) = \frac {1}{\sigma_ {t} ^ {2}} \left(- \tilde {s} _ {t} (s) + \sum_ {a \in \mathcal {A}} a \cdot \phi_ {t} (a, \tilde {s} _ {t} (s))\right), \tag {33}
$$
$$
\phi_ {t} (a, \tilde {s} _ {t} (s)) := \frac {\exp \left\{- \frac {\left| \tilde {s} _ {t} (s) - \gamma_ {t} a \right| ^ {2}}{\sigma_ {t} ^ {2}} \right\}}{\sum_ {\bar {a} \in \mathcal {A}} \exp \left\{- \frac {\left| \tilde {s} _ {t} (s) - \gamma_ {t} \bar {a} \right| ^ {2}}{\sigma_ {t} ^ {2}} \right\}}. \tag {34}
$$
Proof. We will use Lemma 1 to compute the score, for which the only quantity we need to compute is the conditional expectation. Let  $\mathsf{s}' \coloneqq \gamma_t\mathsf{s}$ . Since  $\mathsf{s}$  is discrete,  $P_{\mathsf{s}'}$  is also a uniform distribution over
the symbol set  $\mathcal{A}_t\coloneqq \{\gamma_ta\mid a\in \mathcal{A}\}$ . By Bayes' rule,
$$
\begin{array}{l} \mathbb {P} \left[ \mathbf {s} ^ {\prime} = \gamma_ {t} a \mid \mathbf {s} ^ {\prime} + \sigma_ {t} z _ {s} = x \right] = \frac {\mathbb {P} \left[ \mathbf {s} ^ {\prime} + \sigma_ {t} z _ {s} = x \mid \mathbf {s} ^ {\prime} = \gamma_ {t} a \right] \mathbb {P} \left[ \mathbf {s} ^ {\prime} = \gamma_ {t} a \right]}{\sum_ {\bar {a} \in \mathcal {A}} \mathbb {P} \left[ \mathbf {s} ^ {\prime} + \sigma_ {t} z _ {s} = x \mid \mathbf {s} ^ {\prime} = \gamma_ {t} \bar {a} \right] \mathbb {P} \left[ \mathbf {s} ^ {\prime} = \gamma_ {t} \bar {a} \right]} (35) \\ = \frac {\mathbb {P} \left[ \mathbf {z} _ {s} = \frac {x - \gamma_ {t} a}{\sigma_ {t}} \right] \cdot \frac {1}{| \mathcal {A} |}}{\sum_ {\bar {a} \in \mathcal {A}} \mathbb {P} \left[ \mathbf {z} _ {s} = \frac {x - \gamma_ {t} \bar {a}}{\sigma_ {t}} \right] \cdot \frac {1}{| \mathcal {A} |}} (36) \\ = \frac {\exp \left\{- \frac {\left| x - \gamma_ {t} a \right| ^ {2}}{\sigma_ {t} ^ {2}} \right\}}{\sum_ {\bar {a} \in \mathcal {A}} \exp \left\{- \frac {\left| x - \gamma_ {t} \bar {a} \right| ^ {2}}{\sigma_ {t} ^ {2}} \right\}} (37) \\ = \phi_ {t} (a, x). (38) \\ \end{array}
$$
Therefore, from Lemma 1,
$$
\nabla_ {(\tilde {s} _ {t} (s))} \log p _ {\tilde {\mathbf {s}} _ {t}} (\tilde {s} _ {t} (s)) = \frac {1}{\sigma_ {t} ^ {2}} \left(- \tilde {s} _ {t} (s) + \sum_ {a \in \mathcal {A}} a \cdot \phi_ {t} (a, \tilde {s} _ {t} (s))\right)
$$
As an illustrative example, consider a BPSK source with modes at  $-1$  and  $+1$ . To solve (18) for  $\theta$ , we require the score of the BPSK source at different non-zero levels of Gaussian smoothing. Note that since the source is discrete, the score is undefined if no smoothing is applied. Though this smoothened score can be computed using (33), the stationary points of (18) cannot be computed in closed-form. Hence, in order to study the behavior of the solutions, we simulate (16), as shown in Figure 4. The colored curves plot the loss for a single noise level, i.e., when  $T = 1$ , while the black curve computes the loss across multiple levels of smoothing, by computing an average over the single noise level curves. It is evident from these curves, that while the minima might not lie at the modes  $-1$  and  $+1$  for the single noise level curves, by averaging across multiple noise levels, the solution(s) to (18) approach the mode(s) of the source distribution. As pointed out in §4, an important caveat
is that this behavior is only observed for a good choice of the noise level range. Larger noise levels aid in moving between the modes and smaller noise levels help resolving the solution. Only using larger noise levels, for example, can smoothen out the landscape to the point that the modes are no longer discernible, thus resulting in erroneous estimates. We leverage the noise levels from pre-trained diffusion models that have learned the statistical structure sufficiently well, thus avoiding cumbersome tuning of the noise level range.
Figure 4: Simulating (16) on a BPSK source. The loss at individual timestamps is visualized in addition to the total loss. The minima are at the modes of the BPSK source distribution,  $-1$  and  $+1$ . Larger noise levels allow for exploration between modes and smaller noise levels sharpen the mode-seeking behavior.
B.3 Gaussian Mixture Source Model
Finally, we consider a scalar Gaussian mixture model (GMM) that can often model arbitrary complicated scalar distributions. We derive the score for a 2-component GMM and then generalize the result to an arbitrary  $K$ -component GMM.
Proposition 4. Consider a two-component scalar Gaussian mixture source,
$$
p _ {\mathbf {s}} (s) = \lambda \mathcal {N} \left(s; \mu_ {1}, \sigma_ {1} ^ {2}\right) + (1 - \lambda) \mathcal {N} \left(s; \mu_ {2}, \sigma_ {2} ^ {2}\right), \quad \lambda \in (0, 1). \tag {39}
$$
This only corresponds to choosing an appropriate lower bound on  $1 - \alpha_{T}$  and should not be confused with tuning a noise schedule.
Under the Gaussian smoothing model in (17),
$$
\begin{array}{l} \nabla_ {\tilde {s} _ {t} (s)} \log p _ {\tilde {s} _ {t}} (\tilde {s} _ {t} (s)) = \frac {1}{Z (\tilde {s} _ {t} (s))} \left[ \lambda c _ {1} (\tilde {s} _ {t} (s)) \left(\frac {\gamma_ {t} \mu_ {1} - \tilde {s} _ {t} (s)}{\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}}\right) \right. \\ \left. + (1 - \lambda) c _ {2} \left(\tilde {s} _ {t} (s)\right) \left(\frac {\gamma_ {t} \mu_ {2} - \tilde {s} _ {t} (s)}{\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}}\right) \right], \tag {40} \\ \end{array}
$$
$$
c _ {1} \left(\tilde {s} _ {t} (s)\right) := \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {1}, \gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}\right), \tag {41}
$$
$$
c _ {2} \left(\tilde {s} _ {t} (s)\right) := \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {2}, \gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}\right), \tag {42}
$$
$$
Z \left(\tilde {s} _ {t} (s)\right) := \lambda c _ {1} \left(\tilde {s} _ {t} (s)\right) + (1 - \lambda) c _ {2} \left(\tilde {s} _ {t} (s)\right). \tag {43}
$$
Proof. We need to compute the score of the following distribution,
$$
p _ {\tilde {s} _ {t}} \left(\tilde {s} _ {t} (s)\right) = p _ {\gamma_ {t} \mathbf {s} + \sigma_ {t} \mathbf {z}} \left(\gamma_ {t} \mathbf {s} + \sigma_ {t} \mathbf {z} _ {s} = \tilde {s} _ {t} (s)\right), \tag {44}
$$
Our goal, to this end, is to leverage Lemma 1 to compute the score, thereby requiring an expression for the conditional expectation in (31). Since  $s$  and  $z_{s}$  are independent, (44) can be written as a convolution between two distributions,
$$
p _ {\bar {s} _ {t}} \left(\tilde {s} _ {t} (s)\right) = \left(p _ {\mathbf {s} ^ {\prime}} * p _ {\mathbf {z} _ {\sigma_ {t}}}\right) \left(\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s)\right), \tag {45}
$$
$$
\mathsf {s} ^ {\prime} := \gamma_ {t} \mathsf {s} \quad \text {a n d} \quad \mathsf {z} _ {\sigma_ {t}} := \sigma_ {t} \mathsf {z} _ {s}. \tag {46}
$$
In order to compute the conditional expectation, we first show that,
$$
\begin{array}{l} p _ {\mathfrak {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} | \mathfrak {s} ^ {\prime}} (\mathfrak {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) | \mathfrak {s} ^ {\prime} = a ^ {\prime}) p _ {\mathfrak {s} ^ {\prime}} (\mathfrak {s} ^ {\prime} = a ^ {\prime}) \\ = \frac {1}{Z (\tilde {s} _ {t} (s))} \left[ \lambda c _ {1} (\tilde {s} _ {t} (s)) \mathcal {N} \left(a; \frac {\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} \tilde {s} _ {t} (s) + \gamma_ {t} \sigma_ {t} ^ {2} \mu_ {1}}{\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}}, \frac {\gamma_ {t} ^ {2} \sigma_ {t} ^ {2} \sigma_ {1} ^ {2}}{\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}}\right) \right. \\ \left. + (1 - \lambda) c _ {2} \left(\tilde {s} _ {t} (s)\right) \mathcal {N} \left(a; \frac {\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} \tilde {s} _ {t} (s) + \gamma_ {t} \sigma_ {t} ^ {2} \mu_ {2}}{\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}}, \frac {\gamma_ {t} ^ {2} \sigma_ {t} ^ {2} \sigma_ {2} ^ {2}}{\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}}\right) \right], \tag {47} \\ \end{array}
$$
$$
c _ {1} \left(\tilde {s} _ {t} (s)\right) := \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {1}, \gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}\right),
$$
$$
c _ {2} \left(\tilde {s} _ {t} (s)\right) := \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {2}, \gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}\right),
$$
$$
Z (\tilde {s} _ {t} (s)) := \lambda c _ {1} (\tilde {s} _ {t} (s)) + (1 - \lambda) c _ {2} (\tilde {s} _ {t} (s)).
$$
We start by deriving the density functions for the random variables in (46). Since the convolution between two Gaussians is a Gaussian,
$$
p _ {s ^ {\prime}} (a) = \lambda \mathcal {N} \left(a; \gamma_ {t} \mu_ {1}, \gamma_ {t} ^ {2} \sigma_ {1} ^ {2}\right) + (1 - \lambda) \mathcal {N} \left(a; \gamma_ {t} \mu_ {2}, \gamma_ {t} ^ {2} \sigma_ {2} ^ {2}\right), \tag {48}
$$
$$
p _ {\mathbf {z} _ {\sigma_ {t}}} (w) = \mathcal {N} (w; 0, \sigma_ {t} ^ {2}), \tag {49}
$$
from which it follows, for example through the identities derived in [48, Sec 1]), that
$$
p _ {\tilde {s} _ {t}} \left(\tilde {s} _ {t} (s)\right) = \lambda \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {1}, \gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}\right) + (1 - \lambda) \mathcal {N} \left(\tilde {s} _ {t} (s); \gamma_ {t} \mu_ {2}, \gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}\right). \tag {50}
$$
We conclude the derivation of the conditional distribution by using Bayes' rule,
$$
p _ {\mathbf {s} ^ {\prime} | \mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}}} \left(\mathbf {s} ^ {\prime} = a | \mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s)\right) = \frac {p _ {\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} | \mathbf {s} ^ {\prime}} \left(\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) | \mathbf {s} ^ {\prime} = a\right) p _ {\mathbf {s} ^ {\prime}} \left(\mathbf {s} ^ {\prime} = a\right)}{\int p _ {\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} | \mathbf {s} ^ {\prime}} \left(\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) | \mathbf {s} ^ {\prime} = a ^ {\prime}\right) p _ {\mathbf {s} ^ {\prime}} \left(\mathbf {s} ^ {\prime} = a ^ {\prime}\right) d a ^ {\prime}}. \tag {51}
$$
whereby simplifying the terms in the numerator further, we have
$$
p _ {\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} | \mathbf {s} ^ {\prime}} \left(\mathbf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) \mid \mathbf {s} ^ {\prime} = a\right) = p _ {\mathbf {z} _ {\sigma_ {t}}} \left(\mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) - a\right) = \mathcal {N} \left(a; \tilde {s} _ {t} (s), \sigma_ {t} ^ {2}\right). \tag {52}
$$
Finally, by following the Gaussian product identities in [48, Sec 1], between the distribution in (52) and (48) we reach the result in (47).
Figure 5: Left: A GMM source with two equiprobable modes at  $-1$  and  $-2$ . Two smaller modes are present at  $+2$  and  $+4$ . Right: The minima of (16) lie (approximately) at the modes of the source distribution (black curve). Colored curves correspond to (16) evaluated with  $T = 1$ .
Now we can compute the expectation as,
$$
\begin{array}{l} \mathbb {E} \left[ \mathsf {s} ^ {\prime} | \mathsf {s} ^ {\prime} + \mathbf {z} _ {\sigma_ {t}} = \tilde {s} _ {t} (s) \right] = \frac {1}{Z (\tilde {s} _ {t} (s))} \left[ \lambda c _ {1} (\tilde {s} _ {t} (s)) \left(\frac {\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} \tilde {s} _ {t} (s) + \gamma_ {t} \sigma_ {t} ^ {2} \mu_ {1}}{\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}}\right) \right. \\ \left. + \left(1 - \lambda\right) c _ {2} \left(\tilde {s} _ {t} (s)\right) \left(\frac {\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} \tilde {s} _ {t} (s) + \gamma_ {t} \sigma_ {t} ^ {2} \mu_ {2}}{\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}}\right) \right]. \tag {53} \\ \end{array}
$$
Using Lemma 1 we can express the score as,
$$
\begin{array}{l} \nabla_ {\tilde {s} _ {t} (s)} \log p _ {\tilde {s} _ {t}} (\tilde {s} _ {t} (s)) = \frac {1}{Z (\tilde {s} _ {t} (s))} \left[ \lambda c _ {1} (\tilde {s} _ {t} (s)) \left(\frac {\gamma_ {t} \mu_ {1} - \tilde {s} _ {t} (s)}{\gamma_ {t} ^ {2} \sigma_ {1} ^ {2} + \sigma_ {t} ^ {2}}\right) \right. \\ \left. + (1 - \lambda) c _ {2} (\tilde {s} _ {t} (s)) \left(\frac {\gamma_ {t} \mu_ {2} - \tilde {s} _ {t} (s)}{\gamma_ {t} ^ {2} \sigma_ {2} ^ {2} + \sigma_ {t} ^ {2}}\right) \right]. \\ \end{array}
$$
We can generalize the score to an arbitrary  $K$ -component GMM by exploiting symmetries in (40).
Proposition 5. Consider a  $K$ -component Gaussian mixture source,