SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For a given rate  $\mathsf{R}$ , the following corollary provides the optimization programs which lead to the characterization of the DP tradeoff  $\mathcal{DP}(\mathsf{R})$  for the Gauss-Markov source model.
Corollary 1 For a given rate tuple  $\mathsf{R}$  and  $T = 2$  frames, the optimal reconstructions of the DP-tradeoff  $\mathcal{DP}(\mathsf{R})$  can be written as follows
$$
\hat {X} _ {1} ^ {G} = \nu X _ {1} + Z _ {1}, \tag {186}
$$
$$
\hat {X} _ {2} ^ {G} = \omega_ {1} \hat {X} _ {1} ^ {G} + \omega_ {2} X _ {2} + Z _ {2}, \tag {187}
$$
where  $Z_{1}$  (resp  $Z_{2}$ ) is a Gaussian random variable independent of  $X_{1}$  (resp  $(\hat{X}_1^G, X_2)$ ) and  $\hat{X}_j^G \sim \mathcal{N}(0, \hat{\sigma}_j^2)$  for  $j = 1, 2$ , and  $\nu, \omega_1, \omega_2, \hat{\sigma}_1^2, \hat{\sigma}_2^2$  are the solutions of the following optimization program for the first step,
$$
\min  _ {\nu , \hat {\sigma} _ {1} ^ {2}} \sigma_ {1} ^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \nu \sigma_ {1} ^ {2}, \tag {188a}
$$
$$
s. t. \quad \nu^ {2} \sigma_ {1} ^ {2} \leq \hat {\sigma} _ {1} ^ {2} \left(1 - 2 ^ {- 2 R _ {1}}\right), \tag {188b}
$$
$$
\left(\sigma_ {1} - \hat {\sigma} _ {1}\right) ^ {2} \leq P _ {1}, \tag {188c}
$$
and the following minimization problem for the second step and PLF-FMD,
$$
\min  _ {\omega_ {1}, \omega_ {2}, \hat {\sigma} _ {2} ^ {2}} \sigma_ {2} ^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \nu \omega_ {1} \rho_ {1} \sigma_ {1} \sigma_ {2} - 2 \omega_ {2} \sigma_ {2} ^ {2}, \tag {189a}
$$
$$
s. t. \quad \omega_ {2} ^ {2} \sigma_ {2} ^ {2} \left(1 - 2 ^ {- 2 R _ {2}} \frac {\nu^ {2} \rho_ {1} ^ {2} \sigma_ {1} ^ {2}}{\hat {\sigma} _ {1} ^ {2}}\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \nu \rho_ {1} \sigma_ {1} \sigma_ {2}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right), (1 8 9 b)
$$
$$
\left(\sigma_ {2} - \hat {\sigma} _ {2}\right) ^ {2} \leq P _ {2}, \tag {189c}
$$
or the following minimization problem for the second step and PLF-JD,
$$
\min  _ {\omega_ {1}, \omega_ {2}, \hat {\sigma} _ {2} ^ {2}} \sigma_ {2} ^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \nu \omega_ {1} \rho_ {1} \sigma_ {1} \sigma_ {2} - 2 \omega_ {2} \sigma_ {2} ^ {2} \tag {190a}
$$
$$
s. t. \quad \omega_ {2} ^ {2} \sigma_ {2} ^ {2} \left(1 - 2 ^ {- 2 R _ {2}} \frac {\nu^ {2} \rho_ {1} ^ {2} \sigma_ {1} ^ {2}}{\hat {\sigma} _ {1} ^ {2}}\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \nu \rho_ {1} \sigma_ {1} \sigma_ {2}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right), (1 9 0 b)
$$
$$
\left. t r \left(\Sigma_ {1 2} + \hat {\Sigma} _ {1 2} - 2 \left(\Sigma_ {1 2} ^ {1 / 2} \hat {\Sigma} _ {1 2} \Sigma_ {1 2} ^ {1 / 2}\right) ^ {1 / 2}\right) \leq P _ {2}, \right. \tag {190c}
$$
where  $tr(.)$  denotes the trace of a matrix and
$$
\Sigma_ {1 2} := \left( \begin{array}{c c} \sigma_ {1} ^ {2} & \rho_ {1} \sigma_ {1} \sigma_ {2} \\ \rho_ {1} \sigma_ {1} \sigma_ {2} & \sigma_ {2} ^ {2} \end{array} \right), \tag {191}
$$
$$
\hat {\Sigma} _ {1 2} := \left( \begin{array}{c c} \hat {\sigma} _ {1} ^ {2} & \omega_ {1} \hat {\sigma} _ {1} ^ {2} + \nu \omega_ {2} \rho_ {1} \sigma_ {1} \sigma_ {2} \\ \omega_ {1} \hat {\sigma} _ {1} ^ {2} + \nu \omega_ {2} \rho_ {1} \sigma_ {1} \sigma_ {2} & \hat {\sigma} _ {2} ^ {2} \end{array} \right). \tag {192}
$$
Proof: We obtain the optimization programs for  $T = 2$  frames as follows.
For a given rate tuple  $\mathsf{R}$ , the DP-tradeoff  $\mathcal{DP}(\mathsf{R})$  is given by the set of all tuples  $(\mathrm{D},\mathrm{P})$  such that there exists  $\hat{\mathbf{X}}^G$  satisfying the following Markov chains
$$
\hat {X} _ {1} ^ {G} \rightarrow X _ {1} \rightarrow X _ {2}, \tag {193}
$$
$$
\hat {X} _ {2} ^ {G} \rightarrow \left(\hat {X} _ {1} ^ {G}, X _ {2}\right)\rightarrow X _ {1}, \tag {194}
$$
and the following conditions,
$$
R _ {1} \geq I \left(X _ {1}; \hat {X} _ {1} ^ {G}\right), \tag {195}
$$
$$
R _ {2} \geq I \left(X _ {2}; \hat {X} _ {2} ^ {G} \mid \hat {X} _ {1} ^ {G}\right), \tag {196}
$$
$$
D _ {j} \geq \mathbb {E} \left[ \left\| X _ {j} - \hat {X} _ {j} ^ {G} \right\| ^ {2} \right], \quad j = 1, 2, \tag {197}
$$
$$
P _ {j} \geq W _ {2} ^ {2} \left(P _ {X _ {1} \dots X _ {j}}, P _ {\hat {X} _ {1} ^ {G} \dots \hat {X} _ {j} ^ {G}}\right). \tag {198}
$$
In general, the set of reconstructions that satisfy (193)-(194) can be written as follows
$$
\hat {X} _ {1} ^ {G} = \nu X _ {1} + Z _ {1}, \tag {199}
$$
$$
\hat {X} _ {2} ^ {G} = \omega_ {1} \hat {X} _ {1} ^ {G} + \omega_ {2} X _ {2} + Z _ {2}. \tag {200}
$$
Plugging the above into (195) and (196) yields the following rate expressions
$$
\frac {1}{2} \log \frac {\hat {\sigma} _ {1} ^ {2}}{\hat {\sigma} _ {1} ^ {2} - \nu^ {2} \sigma_ {1} ^ {2}} \leq R _ {1}, \tag {201}
$$
$$
\frac {1}{2} \log \frac {\hat {\sigma} _ {2} ^ {2} - \left(\omega_ {1} \hat {\sigma} _ {1} + \frac {\omega_ {2} \nu \rho_ {1} \sigma_ {1} \sigma_ {2}}{\hat {\sigma} _ {1}}\right) ^ {2}}{\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - \omega_ {2} ^ {2} \sigma_ {2} ^ {2} - 2 \omega_ {1} \omega_ {2} \nu \rho_ {1} \sigma_ {1} \sigma_ {2}} \leq R _ {2}. \tag {202}
$$
Re-arranging the terms in the above constraints yields the conditions in (188b) and (190b). Considering (197) with (199)-(200) gives the following expressions for distortions
$$
\mathbb {E} \left[ \| X _ {1} - \hat {X} _ {1} ^ {G} \| ^ {2} \right] = \sigma_ {1} ^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \mathbb {E} \left[ X _ {1} \hat {X} _ {1} ^ {G} \right] = \sigma_ {1} ^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \nu \sigma_ {1} ^ {2}, \tag {203}
$$
$$
\mathbb {E} \left[ \| X _ {2} - \hat {X} _ {2} ^ {G} \| ^ {2} \right] = \sigma_ {2} ^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \mathbb {E} \left[ X _ {2} \hat {X} _ {2} ^ {G} \right] = \sigma_ {2} ^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \omega_ {1} \nu \rho_ {1} \sigma_ {1} \sigma_ {2} - 2 \omega_ {2} \sigma_ {2} ^ {2}, \tag {204}
$$
which are the objective functions in (188a) and (190a). Now, we evaluate the perception constraint. Notice that the covariance matrices of  $(X_{1},X_{2})$  and  $(\hat{X}_1^G,\hat{X}_2^G)$  are given by  $\Sigma_{12}$  and  $\hat{\Sigma}_{12}$  defined in (191) and (192), respectively. The Wasserstein-2 distance between two Gaussian distributions with covariance matrices  $\Sigma_{12}$  and  $\hat{\Sigma}_{12}$  is given in (190c) as discussed in [26, pp. 18].
Similarly, the expressions in (189) for the decoder based on PLF-FMD can be obtained.
F Gauss-Markov Source Model: Extremal Rates
In this section, we derive the achievable reconstructions for some special cases. We assume that we have only two frames, i.e.,  $D_{3}, P_{3} \to \infty$ . Moreover, let  $\sigma_{1}^{2} = \sigma_{2}^{2} \coloneqq \sigma^{2}$  for simplicity. In general, the reconstructions can be written as follows
$$
\hat {X} _ {1} ^ {G} = \nu X _ {1} + Z _ {1}, \tag {205}
$$
$$
\hat {X} _ {2} ^ {G} = \omega_ {1} \hat {X} _ {1} ^ {G} + \omega_ {2} X _ {2} + Z _ {2}, \tag {206}
$$
where  $\hat{X}_j^G\sim \mathcal{N}(0,\hat{\sigma}_j^2)$  for  $j = 1,2$ . Recall the optimization program of the first step in (188) as follows
$$
\min  _ {\nu , \hat {\sigma} _ {1} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \nu \sigma^ {2}, \tag {207a}
$$
$$
\text {s . t .} \quad \nu^ {2} \sigma^ {2} \leq \hat {\sigma} _ {1} ^ {2} \left(1 - 2 ^ {- 2 R _ {1}}\right), \tag {207b}
$$
$$
\left(\sigma - \hat {\sigma} _ {1}\right) ^ {2} \leq P _ {1}, \tag {207c}
$$
For a given  $\hat{\sigma}_1^2$ , the objective function in (207a) is a monotonically deacreasing function of  $\nu$ , hence one can restrict  $\nu$  to be nonnegative, without loss of optimality. So, the above optimization program can be written as
$$
\min  _ {\nu , \hat {\sigma} _ {1} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \nu \sigma^ {2}, \tag {208a}
$$
$$
0 \leq \nu \leq \frac {\hat {\sigma} _ {1}}{\sigma} \sqrt {1 - 2 ^ {- 2 R _ {1}}}, \tag {208b}
$$
$$
\left(\sigma - \hat {\sigma} _ {1}\right) ^ {2} \leq P _ {1}, \tag {208c}
$$
Optimizing with respect to  $\nu$  in the above program, we have
$$
\nu = \frac {\hat {\sigma} _ {1}}{\sigma} \sqrt {1 - 2 ^ {- 2 R _ {1}}}, \tag {209}
$$
where the optimization program reduces to
$$
\min  _ {\hat {\sigma} _ {1} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \sigma \hat {\sigma} _ {1} \sqrt {1 - 2 ^ {- 2 R _ {1}}}, \tag {210a}
$$
$$
s. t. \quad (\sigma - \hat {\sigma} _ {1}) ^ {2} \leq P _ {1}. \tag {210b}
$$
Next, recall the optimization program of the second step for PLF-FMD in (189) as follows
$$
\min  _ {\omega_ {1}, \omega_ {2}, \hat {\sigma} _ {2} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \nu \omega_ {1} \rho_ {1} \sigma^ {2} - 2 \omega_ {2} \sigma^ {2}, \tag {211a}
$$
$$
\omega_ {2} ^ {2} \sigma^ {2} \left(1 - 2 ^ {- 2 R _ {2}} \frac {\nu^ {2} \rho_ {1} ^ {2} \sigma^ {2}}{\hat {\sigma} _ {1} ^ {2}}\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \nu \rho_ {1} \sigma^ {2}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right), \tag {211b}
$$
$$
\left(\sigma - \hat {\sigma} _ {2}\right) ^ {2} \leq P _ {2}, \tag {211c}
$$
Plugging (209) into the above program, we get
$$
\min  _ {\omega_ {1}, \omega_ {2}, \hat {\sigma} _ {2} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \omega_ {1} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {1 - 2 ^ {- 2 R _ {1}}} - 2 \omega_ {2} \sigma^ {2}, \tag {212a}
$$
$$
\omega_ {2} ^ {2} \sigma^ {2} \left(1 - \rho_ {1} ^ {2} 2 ^ {- 2 R _ {2}} \left(1 - 2 ^ {- 2 R _ {1}}\right)\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {1 - 2 ^ {- 2 R _ {1}}}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right), \tag {212b}
$$
$$
\left(\sigma - \hat {\sigma} _ {2}\right) ^ {2} \leq P _ {2}, \tag {212c}
$$
The optimization program for the second step of PLF-JD is similar to the above program (212) when (212c) is replaced by (190c). In this section, we study different rate regimes and obtain the solutions of the above optimization programs. In particular, we are interested in two perception thresholds  $P_{2} \to \infty$  and  $P_{2} = 0$  where the former corresponds to the classical rate-distortion region and the latter is the case of 0-PLF. For the 0-PLF-FMD, we have  $\hat{\sigma}_{1} = \hat{\sigma}_{2} = \sigma$ . For the 0-PLF-JD, in addition to preserving the marginals, the correlation  $\mathbb{E}[\hat{X}_1^G \hat{X}_2^G] = \rho_1 \sigma^2$  should be satisfied. For each of these cases, the optimization program in (212) is simplified in the following.
Optimization Program of the Second Step for  $P \to \infty$ : In this case, there is no perception constraint in the setting and the optimization program in (212) reduces to the following
$$
\min  _ {\hat {\sigma} _ {2} ^ {2}, \omega_ {1}, \omega_ {2}} \sigma^ {2} + \hat {\sigma} _ {2} ^ {2} - 2 \omega_ {1} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {1 - 2 ^ {- 2 R _ {1}}} - 2 \omega_ {2} \sigma^ {2}, \tag {213a}
$$
$$
\omega_ {2} ^ {2} \sigma^ {2} \left(1 - \rho_ {1} ^ {2} 2 ^ {- 2 R _ {2}} \left(1 - 2 ^ {- 2 R _ {1}}\right)\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {1 - 2 ^ {- 2 R _ {1}}}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right). \tag {213b}
$$
This case corresponds to the classical rate-distortion tradeoff where it is shown that for a given rate, the MMSE reconstructions are indeed optimal [28, 37]. The expressions for MMSE reconstructions are given in Appendix H.1.
Optimization Program of the Second Step for 0-PLF-FMD: In this case, we have  $\hat{\sigma}_1 = \hat{\sigma}_2 = \sigma$ . So, the optimization program in (212) reduces to the following
$$
\min  _ {\omega_ {1}, \omega_ {2}} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} \sqrt {1 - 2 ^ {- 2 R _ {1}}} - 2 \omega_ {2} \sigma^ {2}, \tag {214a}
$$
$$
\omega_ {2} ^ {2} \left(1 - \rho_ {1} ^ {2} 2 ^ {- 2 R _ {2}} \left(1 - 2 ^ {- 2 R _ {1}}\right)\right) \leq \left(1 - \omega_ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1} \sqrt {1 - 2 ^ {- 2 R _ {1}}}\right) \left(1 - 2 ^ {- 2 R _ {2}}\right). \tag {214b}
$$
Here,  $\omega_{1}$  and  $\omega_{2}$  only need to satisfy the rate constraint given in (214b) which represents a larger search space than that of 0-PLF-JD which will be discussed in the following.
Optimization Program of the Second Step for 0-PLF-JD: In this case, in addition to preserving marginals  $\hat{\sigma}_1 = \hat{\sigma}_2 = \sigma$ , we need to satisfy the constraint  $\mathbb{E}[\hat{X}_1^G\hat{X}_2^G] = \rho_1\sigma^2$ . Thus, the optimization program of this case has an extra condition  $\omega_{1} + \nu \omega_{2}\rho_{1} = \rho_{1}$  comparing to (214) and it is given as follows
$$
\min  _ {\omega_ {1}, \omega_ {2}} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} \sqrt {1 - 2 ^ {- 2 R _ {1}}} - 2 \omega_ {2} \sigma^ {2}, \tag {215a}
$$
s.t.  $\omega_{2}^{2}(1 - \rho_{1}^{2}2^{-2R_{2}}(1 - 2^{-2R_{1}}))\leq (1 - \omega_{1}^{2} - 2\omega_{1}\omega_{2}\rho_{1}\sqrt{1 - 2^{-2R_{1}}})(1 - 2^{-2R_{2}}),$
$$
\omega_ {1} + \nu \omega_ {2} \rho_ {1} = \rho_ {1}. \tag {215b}
$$
Comparing (215) with (214), we notice that the search space of the optimization program for 0-PLF-JD is smaller than that of 0-PLF-FMD. Thus, a larger distortion is expected for 0-PLF-JD.
Before studying each case of extremal rates, we introduce another constraint in the optimization program of all above three cases of perception metrics. We restrict to nonnegative  $\omega_{1}\omega_{2}\rho_{1}$  and get an upper bound on the programs (213), (214) and (215). So, in further discussion on these programs, the constraint  $\omega_{1}\omega_{2}\rho_{1} \geq 0$  will be also considered.
1)  $R_{1} = R_{2} = \epsilon$  for small  $\epsilon$ :
In the low-rate regime, notice that we can approximate the rate term as follows
$$
1 - 2 ^ {- 2 \epsilon} = 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right). \tag {216}
$$
Plugging the above into (209), we have
$$
\nu = \frac {\hat {\sigma} _ {1}}{\sigma} \sqrt {2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)}. \tag {217}
$$
Also, inserting (216) into the rate constraint of the second step (211c) yields the following
$$
\omega_ {2} ^ {2} \sigma^ {2} \left(1 - \rho_ {1} ^ {2} 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right) \leq \left(\hat {\sigma} _ {2} ^ {2} - \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)}\right) \left(2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right). ^ {2} \tag {218}
$$
Re-arranging the terms in the above inequality yields the following
$$
\begin{array}{l} \hat {\sigma} _ {2} ^ {2} \geq \frac {\omega_ {2} ^ {2} \sigma^ {2} \left(1 - \rho_ {1} ^ {2} 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right)}{2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)} + \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)} (219) \\ = \omega_ {2} ^ {2} \sigma^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})} (220) \\ \end{array}
$$
So, in all of the optimization programs of the case  $R_{1} = R_{2} = \epsilon$ , the above constraint (220) will replace the rate constraint of the second step.
Now, we consider different cases based on the perception measure.
a) Without a perception constraint: In this case, using (216), the optimization program of the first step in (210) simplifies to the following
$$
D _ {1} = \min  _ {\hat {\sigma} _ {1} ^ {2}} \sigma^ {2} + \hat {\sigma} _ {1} ^ {2} - 2 \sigma \hat {\sigma} _ {1} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})}, \tag {221}
$$
which gives us the following optimal solution
$$
\hat {\sigma} _ {1} = \sqrt {2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)} \sigma = \sqrt {2 \epsilon \ln 2} \sigma + O (\epsilon). \tag {222}
$$
Plugging the above solution into (217) and (221), we get
$$
\nu = 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right), \tag {223}
$$
2The inequalities of the form  $f(\epsilon) + O(\epsilon^2)\leq g(\epsilon) + O(\epsilon^2)$  , where  $f(\epsilon),g(\epsilon) = \Omega (\epsilon^{2})$  , imply that  $f(\epsilon)\leq g(\epsilon)$  . So, in such inequalities, we work with dominant terms  $(f(\epsilon),g(\epsilon))$  and ignore the small terms  $O(\epsilon^2)$  . A similar argument holds if we have other orders of  $\epsilon$  and the functions  $f(.),g(.)$  approach zero slower than them.
$$
D _ {1} = (1 - 2 \epsilon \ln 2) \sigma^ {2} + O (\epsilon^ {2}). \tag {224}
$$
Now, we look at the optimization program of the second step (213). For a given  $\omega_{1}$  and  $\omega_{2}$ , the objective function is an increasing function of  $\hat{\sigma}_{2}^{2}$ , so optimizing over  $\hat{\sigma}_{2}^{2}$  yields the following
$$
\hat {\sigma} _ {2} ^ {2} = \omega_ {2} ^ {2} \sigma^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})}. \tag {225}
$$
Thus, the optimization program (213) is further upper bounded by the following
$$
\min  _ {\substack {\hat {\sigma} _ {2} ^ {2}, \omega_ {1}, \omega_ {2}: \\ \omega_ {1} \omega_ {2} \rho_ {1} \geq 0}} \sigma^ {2} + \omega_ {2} ^ {2} \sigma^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} \hat {\sigma} _ {1} ^ {2} - 2 (1 - \omega_ {2}) \omega_ {1} \rho_ {1} \hat {\sigma} _ {1} \sigma \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})} - 2 \omega_ {2} \sigma^ {2}. \tag{226}
$$
The optimal solution of the above minimization is given by the following
$$
\omega_ {1} = \rho_ {1} + O (\epsilon), \tag {227}
$$
$$
\omega_ {2} = 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right). \tag {228}
$$
Thus, considering the dominant terms of (223), (227) and (228), we have
$$
\hat {X} _ {1} ^ {G} = (2 \epsilon \ln 2) X _ {1} + Z _ {1}, \tag {229}
$$
$$
\hat {X} _ {2} ^ {G} = \rho_ {1} \hat {X} _ {1} ^ {G} + (2 \epsilon \ln 2) X _ {2} + Z _ {2}, \tag {230}
$$
and  $Z_{j}\sim \mathcal{N}(0,2\epsilon \sigma^{2}\ln 2)$  for  $j = 1,2$  . Notice that
$$
D _ {1} = (1 - 2 \epsilon \ln 2) \sigma^ {2}, \tag {231}
$$
$$
D _ {2} = \left(1 - \left(1 + \rho_ {1} ^ {2}\right) 2 \epsilon \ln 2\right) \sigma^ {2}. \tag {232}
$$
b) 0-PLF-FMD: In this case, we have  $\hat{\sigma}_1 = \hat{\sigma}_2 = \sigma$ . For the optimization program of the first step, (209) reduces to the following
$$
\nu = \sqrt {2 \epsilon \ln 2} + O (\epsilon), \tag {233}
$$
and  $D_{1}$  is given in the following which is derived by (210)
$$
D _ {1} = 2 \left(1 - \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2} + O (\epsilon). \tag {234}
$$
Now, we study the optimization program of the second step. The optimization program of (214) is further upper bounded by the following
$$
\min  _ {\substack {\omega_ {1}, \omega_ {2}: \\ \omega_ {1} \omega_ {2} \rho_ {1} \geq 0}} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})} - 2 \omega_ {2} \sigma^ {2}, \tag{235a}
$$
$$
\text {s . t .} \quad 1 \geq \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})}}. \tag {235b}
$$
Now, we further simplify the inequality (235b) in the following. Considering the fact that  $\omega_{1}\omega_{2}\rho_{1} \geq 0$ , this inequality implies that
$$
\omega_ {1} ^ {2} \leq 1, \tag {236}
$$
$$
\omega_ {2} ^ {2} \leq 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right). \tag {237}
$$
So, using the above inequalities, the RHS of (235b) can be upper bounded as follows
$$
\begin{array}{l} \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})}} \\ \leq \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + (\omega_ {1} ^ {2} + \omega_ {2} ^ {2}) \rho_ {1} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})}} \\ \leq \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + O \left(\epsilon^ {3 / 2}\right)}. \tag {238} \\ \end{array}
$$
Now, according to (238), the optimization program in (235) is further upper bounded by the following
$$
\min  _ {\substack {\omega_ {1}, \omega_ {2}: \\ \omega_ {1} \omega_ {2} \rho_ {1} \geq 0}} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})} - 2 \omega_ {2} \sigma^ {2}, \tag{239a}
$$
$$
\text {s . t .} \quad 1 \geq \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + O \left(\epsilon^ {3 / 2}\right)}. \tag {239b}
$$
For a given  $\omega_{1}$  (resp  $\omega_{2}$ ), the objective function (239a) is a monotonically decreasing function of  $\omega_{2}$  (resp  $\omega_{1}$ ), so the optimal solution is attained on the boundary, i.e.,
$$
1 = \sqrt {\omega_ {2} ^ {2} \left(\frac {1}{2 \epsilon \ln 2} + O (1)\right) + \omega_ {1} ^ {2} + O \left(\epsilon^ {3 / 2}\right)} \tag {240}
$$
Thus, the program (239) further simplifies to the following
$$
\min  _ {\substack {\omega_ {1}: \\ \omega_ {1} \rho_ {1} \geq 0}} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} \sqrt {2 \epsilon \ln 2 + O (\epsilon^ {2})} - 2 \sigma^ {2} \sqrt {(1 - \omega_ {1} ^ {2} - O (\epsilon^ {3 / 2})) (2 \epsilon \ln 2 + O (\epsilon^ {2}))}. \tag{241}
$$
The optimal solution of the above program is given by
$$
\omega_ {1} = \frac {\rho_ {1}}{\sqrt {1 + \rho_ {1} ^ {2}}} + O (\epsilon), \tag {242}
$$
which together with (240) yields
$$
\omega_ {2} = \sqrt {\frac {2 \epsilon \ln 2}{1 + \rho_ {1} ^ {2}}} + O (\epsilon). \tag {243}
$$
Thus, considering dominant terms of (233), (242) and (243), we get
$$
\hat {X} _ {1} ^ {G} = \sqrt {2 \epsilon \ln 2} X _ {1} + Z _ {1}, \tag {244}
$$
$$
\hat {X} _ {2} ^ {G} = \frac {\rho_ {1}}{\sqrt {1 + \rho_ {1} ^ {2}}} \hat {X} _ {1} ^ {G} + \sqrt {\frac {2 \epsilon \ln 2}{1 + \rho_ {1} ^ {2}}} X _ {2} + Z _ {2}, \tag {245}
$$
where  $Z_{1}\sim \mathcal{N}(0,(1 - 2\epsilon \ln 2)\sigma^{2})$  and
$$
Z _ {2} \sim \mathcal {N} (0, \left(1 - \frac {\rho_ {1} ^ {2}}{1 + \rho_ {1} ^ {2}} - \frac {1 + 2 \rho_ {1} ^ {2}}{1 + \rho_ {1} ^ {2}} 2 \epsilon \ln 2\right) \sigma^ {2}). \tag {246}
$$
$$
D _ {1} = 2 \left(1 - \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \tag {247}
$$
$$
D _ {2} = 2 \left(1 - \sqrt {\left(1 + \rho_ {1} ^ {2}\right) 2 \epsilon \ln 2}\right) \sigma^ {2}. \tag {248}
$$
For the special case of  $\rho_{1} = 1$ , the expressions in (244) and (245) simplify as follows
$$
\hat {X} _ {1} ^ {G} = \sqrt {2 \epsilon \ln 2} X _ {1} + Z _ {1}, \tag {249}
$$
$$
\hat {X} _ {2} ^ {G} = \sqrt {2} \sqrt {2 \epsilon \ln 2} X _ {1} + \frac {1}{\sqrt {2}} Z _ {1} + Z _ {2}. \tag {250}
$$
Define  $Z_{\mathrm{FMD}} \coloneqq \frac{1}{\sqrt{2}} Z_1 + Z_2$  and notice that  $Z_{\mathrm{FMD}} \sim \mathcal{N}(0, (1 - 4\epsilon \ln 2)\sigma^2)$ . Moreover, we have
$$
D _ {1} = 2 \left(1 - \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \tag {251}
$$
$$
D _ {2} = 2 \left(1 - \sqrt {4 \epsilon \ln 2}\right) \sigma^ {2}. \tag {252}
$$
c) 0-PLF-JD: In this case, the optimization program of the first step is similar to the previous case. The optimization program of the second step is given in (215) where the condition  $\omega_{1} + \nu \omega_{2}\rho_{1} = \rho_{1}$  is introduced. According to (233),  $\nu = O(\sqrt{\epsilon})$  which suggests the following form for  $\omega_{1}$
$$
\omega_ {1} = \rho_ {1} - \delta_ {\epsilon}, \tag {253}
$$
for some small  $\delta_{\epsilon}$  that goes to zero as  $\epsilon \to 0$ . The parameter  $\delta_{\epsilon}$  will be determined later. Plugging  $\omega_{1} = \rho_{1} - \delta_{\epsilon}$  into (240), we find out that only the constant term of  $\omega_{1}$  contributes to a dominant term for  $\omega_{2}$  which yields the following