SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\omega_ {2} = \sqrt {2 \epsilon \ln 2 \left(1 - \rho_ {1} ^ {2}\right)} + O (\epsilon). \tag {254}
$$
$$
\hat {X} _ {1} ^ {G} = \sqrt {2 \epsilon \ln 2} X _ {1} + Z _ {1}, \tag {255}
$$
$$
\hat {X} _ {2} ^ {G} = \left(\rho_ {1} - \delta_ {\epsilon}\right) \hat {X} _ {1} ^ {G} + \sqrt {\left(1 - \rho_ {1} ^ {2}\right) 2 \epsilon \ln 2} X _ {2} + Z _ {2}, \tag {256}
$$
Now, applying the constraint  $\mathbb{E}[\hat{X}_1^G\hat{X}_2^G ] = \rho_1\sigma^2$  , we get
$$
\delta_ {\epsilon} = \rho_ {1} \sqrt {1 - \rho_ {1} ^ {2}} (2 \epsilon \ln 2). \tag {257}
$$
However, notice that since  $\delta_{\epsilon} = O(\epsilon)$ , it does not contribute to dominant terms of distortion. So, we can simply represent  $\hat{X}_1^G$  and  $\hat{X}_2^G$  as follows
$$
\hat {X} _ {1} ^ {G} = \sqrt {2 \epsilon \ln 2} X _ {1} + Z _ {1}, \tag {258}
$$
$$
\hat {X} _ {2} ^ {G} = \rho_ {1} \hat {X} _ {1} ^ {G} + \sqrt {\left(1 - \rho_ {1} ^ {2}\right) 2 \epsilon \ln 2} X _ {2} + Z _ {2}, \tag {259}
$$
where  $Z_{1}\sim \mathcal{N}(0,(1 - 2\epsilon \ln 2)\sigma^{2})$  and  $Z_{2}\sim \mathcal{N}(0,(1 - \rho_1^2 -(1 - \rho_1^2 +2\rho_1^2\sqrt{1 - \rho_1^2})2\epsilon \ln 2)\sigma^2)$  The following distortions are also achievable
$$
D _ {1} = 2 \left(1 - \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \tag {260}
$$
$$
D _ {2} = 2 \left(1 - \left(\rho_ {1} ^ {2} + \sqrt {1 - \rho_ {1} ^ {2}}\right) \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}. \tag {261}
$$
For the special case of  $\rho = 1$ , according to (259) and (261), we have  $\hat{X}_2^G = \hat{X}_1^G$  and  $D_2 = D_1$ .
2)  $R_{1} \to \infty$ ,  $R_{2} = \epsilon$  for small  $\epsilon$ : In this case, since  $R_{1} \to \infty$ , we have  $\hat{X}_{1}^{G} = X_{1}$ ,  $D_{1} = 0$ , and we only need to solve the optimization program of the second step. Also, we have the following approximation
$$
1 - 2 ^ {- 2 R _ {2}} = 1 - 2 ^ {- 2 \epsilon} = 2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right). \tag {262}
$$
We consider three different cases based on the perception constraint.
a) Without a perception constraint: In this case, consider the optimization program (213). For a given  $\omega_{1}$  and  $\omega_{2}$ , the objective function is an increasing function of  $\hat{\sigma}_{2}^{2}$ , hence optimizing over  $\hat{\sigma}_{2}^{2}$ , we get
$$
\hat {\sigma} _ {2} ^ {2} = \frac {\omega_ {2} ^ {2} \sigma^ {2} \left(1 - \rho_ {1} ^ {2} + O (\epsilon)\right)}{2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)} + \omega_ {1} ^ {2} \sigma^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \sigma^ {2}. \tag {263}
$$
The program in (213) is further upper bounded by the following
$$
\min  _ {\substack {\omega_ {1}, \omega_ {2}: \\ \omega_ {1} \omega_ {2} \rho_ {1} \geq 0}} \sigma^ {2} + \frac {\omega_ {2} ^ {2} \sigma^ {2} (1 - \rho_ {1} ^ {2} + O (\epsilon))}{2 \epsilon \ln 2 + O (\epsilon^ {2})} + \omega_ {1} ^ {2} \sigma^ {2} + 2 \omega_ {1} \omega_ {2} \rho_ {1} \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} - 2 \omega_ {2} \sigma^ {2}, \tag{264}
$$
The solution of the above optimization program is given by the following
$$
\omega_ {1} = \rho_ {1} - \rho_ {1} (2 \epsilon \ln 2), \tag {265}
$$
$$
\omega_ {2} = 2 \epsilon \ln 2. \tag {266}
$$
$$
\hat {X} _ {1} ^ {G} = X _ {1}, \tag {267}
$$
$$
\hat {X} _ {2} ^ {G} = \left(\rho_ {1} - \rho_ {1} (2 \epsilon \ln 2)\right) X _ {1} + (2 \epsilon \ln 2) X _ {2} + Z _ {2}, \tag {268}
$$
where  $Z_{2}\sim \mathcal{N}(0,(1 - \rho_{1}^{2})\sigma^{2}2\epsilon \ln 2)$ . So, the reconstruction of the second frame closely resembles the first frame. The distortions of the first and second frames are zero and  $(1 - \rho_1^2 -(1 - \rho_1^2)2\epsilon \ln 2)\sigma^2$  respectively.
b) 0-PLF-FMD: In this case,  $\hat{\sigma}_1 = \hat{\sigma}_2 = \sigma$ . Thus, the optimization program in (214) is further upper bounded by the following
$$
\min  _ { \begin{array}{c} \omega_ {1}, \omega_ {2}: \\ \omega_ {1} \omega_ {2} \rho_ {1} \geq 0 \end{array} } 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} - 2 \omega_ {2} \sigma^ {2}, \tag {269a}
$$
$$
s. t. \quad \omega_ {2} ^ {2} \left(1 - \rho_ {1} ^ {2} + O (\epsilon)\right) \leq \left(1 - \omega_ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1}\right) \left(2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right). \tag {269b}
$$
For a given  $\omega_{1}$  (resp  $\omega_{2}$ ), the objective function (269a) is a monotonically decreasing function of  $\omega_{2}$  (resp  $\omega_{1}$ ). So, the optimal solution is attained on the boundary, i.e., (269b) is satisfied with equality given as follows
$$
\omega_ {2} ^ {2} \left(1 - \rho_ {1} ^ {2} + O (\epsilon)\right) = \left(1 - \omega_ {1} ^ {2} - 2 \omega_ {1} \omega_ {2} \rho_ {1}\right) \left(2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right). \tag {270}
$$
It can be easily verified that the first-order terms of  $\omega_{1}$  and  $\omega_{2}$  which optimize the program are 1 and 0, respectively. So, we write  $\omega_{1}$  and  $\omega_{2}$  in the following form
$$
\omega_ {1} = 1 + (2 \epsilon \ln 2) \delta_ {1} + O \left(\epsilon^ {2}\right), \tag {271}
$$
$$
\omega_ {2} = (2 \epsilon \ln 2) \delta_ {2} + O \left(\epsilon^ {2}\right), \tag {272}
$$
for some real  $\delta_1$  and  $\delta_2$ . Plugging the above (271) and (272) into (270) and considering the dominant terms, we get
$$
\delta_ {2} ^ {2} \left(1 - \rho_ {1} ^ {2}\right) = - 2 \delta_ {1} - 2 \rho_ {1} \delta_ {2}. \tag {273}
$$
On the other side, we can write the objective function in (269) as follows
$$
\begin{array}{l} 2 \sigma^ {2} - 2 \omega_ {1} \rho_ {1} \sigma^ {2} - 2 \omega_ {2} \sigma^ {2} \\ = 2 \sigma^ {2} - 2 \rho_ {1} \omega_ {1} \sigma^ {2} - 2 \omega_ {2} \sigma^ {2} + O (\epsilon^ {2}) (274) \\ = 2 \sigma^ {2} - 2 \rho_ {1} \sigma^ {2} - 2 \left(\rho_ {1} \delta_ {1} \sigma^ {2} + \delta_ {2} \sigma^ {2}\right) (2 \epsilon \ln 2) + O \left(\epsilon^ {2}\right) (275) \\ = 2 \sigma^ {2} - 2 \rho_ {1} \sigma^ {2} - \left(- 2 \rho_ {1} ^ {2} \delta_ {2} \sigma^ {2} - \rho_ {1} \left(1 - \rho_ {1} ^ {2}\right) \delta_ {2} ^ {2} + 2 \delta_ {2} \sigma^ {2}\right) (2 \epsilon \ln 2) + O (\epsilon^ {2}). (276) \\ \end{array}
$$
Differentiating the above expression with respect to  $\delta_{2}$  and letting it be zero, we have:
$$
\delta_ {2} = \frac {1}{\rho_ {1}}, \quad \delta_ {1} = - \frac {1 + \rho_ {1} ^ {2}}{2 \rho_ {1} ^ {2}}. \tag {277}
$$
$$
\hat {X} _ {1} ^ {G} = X _ {1}, \tag {278}
$$
$$
\hat {X} _ {2} ^ {G} = \left(1 - \frac {\left(1 + \rho_ {1} ^ {2}\right) 2 \epsilon \ln 2}{2 \rho_ {1} ^ {2}}\right) \hat {X} _ {1} ^ {G} + \frac {2 \epsilon \ln 2}{\rho_ {1}} X _ {2} + Z _ {2}, \tag {279}
$$
where  $Z_{2}\sim \mathcal{N}(0,(\frac{1 - \rho_{1}^{2}}{\rho_{1}^{2}})2\epsilon \ln 2)$ . Again, the reconstruction of the second frame is almost similar to the first frame and the distortion is  $2(1 - \rho_{1} - (\frac{1 - \rho_{1}^{2}}{2\rho_{1}})2\epsilon \ln 2)\sigma^{2}$ .
c) 0-PLF-JD: First consider the case where  $\rho_{1} \neq 1$ . The optimization program is given in (215) where the constraint  $\omega_{1} + \nu \rho_{1}\omega_{2} = \rho_{1}$  is introduced. Notice that  $\omega_{1}$  can be written in the following form
$$
\omega_ {1} = \rho_ {1} + \delta_ {\epsilon}, \tag {280}
$$
for some  $\delta_{\epsilon}$  that goes to zero as  $\epsilon \to 0$ . The parameter  $\delta_{\epsilon}$  will be determined later. Plugging  $\omega_{1} = \rho_{1} + \delta_{\epsilon}$  into (270) yields the following
$$
\omega_ {2} = \sqrt {2 \epsilon \ln 2} + O (\epsilon), \tag {281}
$$
which is derived only through the first-order term of  $\omega_{1}$  which is  $\rho_{1}$ . Now, considering the fact that  $\mathbb{E}[\hat{X}_1^G\hat{X}_2^G ] = \rho_1\sigma^2$ , we obtain
$$
\delta_ {\epsilon} = - \rho_ {1} \sqrt {2 \epsilon \ln 2}. \tag {282}
$$
$$
\hat {X} _ {1} ^ {G} = X _ {1}, \tag {283}
$$
$$
\hat {X} _ {2} ^ {G} = \left(\rho_ {1} - \rho_ {1} \sqrt {2 \epsilon \ln 2}\right) \hat {X} _ {1} ^ {G} + \sqrt {2 \epsilon \ln 2} X _ {2} + Z _ {2}, \tag {284}
$$
where  $Z_{2}\sim \mathcal{N}(0,(1 - \rho_{1}^{2})\sigma^{2})$  . Here, the reconstruction of the second frame closely resembles the first frame. The distortion of the second frame is  $2(1 - \rho_1^2 -(1 - \rho_1^2)\sqrt{2\epsilon\ln 2})\sigma^2$
If  $\rho_{1} = 1$ , we simply have  $\hat{X}_2^G = \hat{X}_1^G = X_1 = X_2$  which can be derived from (283)-(284) by letting  $X_{1} = X_{2}$ .
The analysis for the case of  $R_{1} = \epsilon$  and  $R_{2} \to \infty$  is similar and is omitted for brevity. The results of this section are summarized in Table 2.
Table 2: Achievable reconstructions for extremal rates and different PLFs (The first, second and third rows represent reconstructions corresponding to the MMSE, 0-PLF-FMD and 0-PLF-JD, respectively).
R1=R2=ε	R1→∞, R2=ε	R1=ε, R2=∞
MMSE	X1G=(2εln 2)X1+Z1	X1G=X1	X1G=(2εln 2)X1+Z1
X2G=ρ1X1G+(2εln 2)X2+Z2	X2G=(ρ1-ρ12εln 2)X1G+(2εln 2)X2+Z2	X2G=X2
Zj~N(0,2εσ2ln 2)	Z2~N(0,(1-ρ12)2εσ2ln 2)	Z1~N(0,2εσ2ln 2)
D1=(1-2εln 2)σ2	D1=0	D1=(1-2εln 2)σ2
D2=(1-(1+ρ12)2εln 2)σ2	D2=(1-ρ12-(1-ρ12)2εln 2)σ2	D2=0
0-PLF-FMD	X1G=√2εln 2X1+Z1	X1G=X1	X1G=√2εln 2X1+Z1
X2G=ρ1/√1+ρ12X1G+√2εln 2X2+Z2	X2G=(1-(1+ρ12)2εln 2/2ρ12)X1G+2εln 2X2+Z2	X2G=X2
Z1~N(0,(1-2εln 2)σ2)	Z2~N(0, (1-ρ12/ρ12)2εln 2)	Z1~N(0,(1-2εln 2)σ2)
Z2~N(0,(1-ρ12-1+2ρ12/1+ρ12)2εln 2)σ2)
D1=2(1-√2εln 2)σ2	D1=0	D1=2(1-√2εln 2)σ2
D2=2(1-√(1+ρ12)2εln 2)σ2	D2=2(1-ρ1-(1-ρ12)/2ρ1)2εln 2)σ2	D2=0
0-PLF-ID	X1G=√2εln 2X1+Z1	X1G=X1	X1G=√2εln 2X1+Z1
X2G=ρ1X1G+√(1-ρ12)2εln 2X2+Z2a	X2G=(ρ1-ρ1√2εln 2)X1G+√2εln 2X2+Z2	X2G=ρ1X1G+√1-ρ12X2
Z1~N(0,(1-2εln 2)σ2)	Z2~N(0,(1-ρ12)σ2)	Z1~N(0,(1-2εln 2)σ2)
Z2~N(0,(1-ρ12-(1-ρ12)2εln 2)σ2)
D1=2(1-√2εln 2)σ2	D1=0	D1=2σ2
D2=2(1-(ρ12+√1-ρ12)√2εln 2)σ2	D2=2(1-ρ12-(1-ρ12)√2εln 2)σ2	D2=2(1-√1-ρ12)-ρ12√2εln 2)σ2
As justified in (253)-(259), the coefficient  $\omega_{1}$  (the coefficient of  $\hat{X}_1^G$  in  $\hat{X}_2^G$ ) has some correction terms of  $O(\epsilon)$  which are ignored in the presentation of  $\hat{X}_2^G$  since they do not contribute to dominant terms of distortion.
G Comparison of PLFs in Low-Rate Regime
Theorem 6 For sufficiently small  $\epsilon$ , let  $R_{j} = \epsilon$  and suppose that  $\rho_{j} = \rho$  and  $\sigma_{j} = \sigma$ , for  $j = 1, \ldots, T$ . The achievable distortions  $D_{FMD,j}$  (for 0-PLF-FMD), and  $D_{JD,j}$  (for 0-PLF-JD) are:
$$
D _ {F M D, j} = 2 \left(1 - \Delta_ {F M D, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \quad D _ {J D, j} = 2 \left(1 - \Delta_ {J D, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \tag {285}
$$
Proof: We extend the proof in the previous section for the low-rate regime to  $T$  frames.
Distortion Analysis for 0-PLF-FMD:
We follow similar steps to (233)-(248) for optimization problems of the third and fourth frames and then use induction to derive expressions for  $T$  frames. For simplicity, we assume that  $\rho_{j} = \rho$  for all  $j$ . Notice that in the following proof,  $(\hat{X}_1^G,\hat{X}_2^G)$  are as in (205)-(206) where  $\nu ,\omega_{1}$  and  $\omega_{2}$  are already derived in (233)-(248).
Now, consider the reconstruction of the third frame as follows
$$
\hat {X} _ {3} ^ {G} = \tau_ {1} \hat {X} _ {1} ^ {G} + \tau_ {2} \hat {X} _ {2} ^ {G} + \tau_ {3} X _ {3} + Z _ {3}, \tag {286}
$$
for some  $\tau_1, \tau_2, \tau_3$ , where  $\hat{X}_3^G \sim \mathcal{N}(0, \sigma^2)$  and  $Z_3$  is a Gaussian random variable independent of  $(\hat{X}_1^G, \hat{X}_2^G, X_3)$ . The rate constraint of the third step is given by
$$
R _ {3} \geq I \left(X _ {3}; \hat {X} _ {3} ^ {G} \mid \hat {X} _ {1} ^ {G}, \hat {X} _ {2} ^ {G}\right). \tag {287}
$$
Evaluating the above constraint with the choice of random variables  $(\hat{X}_1^G, \hat{X}_2^G, \hat{X}_3^G)$  and re-arranging the terms, we get
$$
\begin{array}{l} \tau_ {3} ^ {2} \sigma^ {2} \left(1 - 2 ^ {- 2 R _ {3}} \left(\rho^ {4} 2 ^ {- 2 R _ {1} - 2 R _ {2}} + \rho^ {2} \left(1 - \rho^ {2}\right) 2 ^ {- 2 R _ {2}} - \rho^ {2}\right)\right) \leq \\ (1 - 2 ^ {- 2 R _ {3}}) (1 - \tau_ {1} ^ {2} - \tau_ {2} ^ {2} - 2 \tau_ {1} \tau_ {2} \omega_ {1} \nu - 2 \tau_ {1} \tau_ {2} \omega_ {2} \nu \rho - 2 \tau_ {2} \tau_ {3} \omega_ {1} \nu \rho^ {2} - 2 \tau_ {2} \tau_ {3} \omega_ {2} \rho - 2 \tau_ {1} \tau_ {3} \nu \rho^ {2}) \sigma^ {2}. \tag {288} \\ \end{array}
$$
Similar to (240), considering the dominant terms of the above rate constraint and the fact that the solution of the optimization problem is attained when the above inequality is satisfied with "equality", we get
$$
\left(1 - \tau_ {1} ^ {2} - \tau_ {2} ^ {2} + O \left(\epsilon^ {3 / 2}\right)\right) \left(2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right) = \tau_ {3} ^ {2} \left(1 + O (\epsilon)\right). \tag {289}
$$
The distortion can be written as follows
$$
\mathbb {E} [ \| X _ {3} - \hat {X} _ {3} ^ {G} \| ^ {2} ] = 2 \sigma^ {2} - 2 \tau_ {3} \sigma^ {2} - 2 \tau_ {2} \omega_ {2} \rho \sigma^ {2} - 2 \tau_ {2} \omega_ {1} \nu \rho^ {2} \sigma^ {2} - 2 \tau_ {1} \nu \rho^ {2} \sigma^ {2}. (2 9 0)
$$
So, the goal is to solve the following optimization problem for the third step
$$
\min  _ {\tau_ {1}, \tau_ {2}, \tau_ {3}} 2 \sigma^ {2} - 2 \tau_ {3} \sigma^ {2} - 2 \tau_ {2} \omega_ {2} \rho \sigma^ {2} - 2 \tau_ {2} \omega_ {1} \nu \rho^ {2} \sigma^ {2} - 2 \tau_ {1} \nu \rho^ {2} \sigma^ {2} \tag {291}
$$
$$
\text {s . t .}: \quad \left(1 - \tau_ {1} ^ {2} - \tau_ {2} ^ {2} + O \left(\epsilon^ {3 / 2}\right)\right) \left(2 \epsilon \ln 2 + O \left(\epsilon^ {2}\right)\right) = \tau_ {3} ^ {2} \left(1 + O (\epsilon)\right). \tag {292}
$$
We restrict the search space to  $\tau_1, \tau_2, \tau_3 \geq 0$  and get an upper bound to the above optimization program as follows
$$
\min  _ {\tau_ {1}, \tau_ {2}, \tau_ {3} \geq 0} 2 \sigma^ {2} - 2 \tau_ {3} \sigma^ {2} - 2 \tau_ {2} \omega_ {2} \rho \sigma^ {2} - 2 \tau_ {2} \omega_ {1} \nu \rho^ {2} \sigma^ {2} - 2 \tau_ {1} \nu \rho^ {2} \sigma^ {2} \tag {293}
$$
$$
\text {s . t .}: \quad (1 - \tau_ {1} ^ {2} - \tau_ {2} ^ {2} + O (\epsilon^ {3 / 2})) (2 \epsilon \ln 2 + O (\epsilon^ {2})) = \tau_ {3} ^ {2} (1 + O (\epsilon)). \tag {294}
$$
The above optimization problem is equivalent to the following
$$
\begin{array}{l} \min _ {\tau_ {1}, \tau_ {2} \geq 0} \left(2 \sigma^ {2} - 2 \sqrt {\frac {(2 \epsilon \ln 2 + O (\epsilon^ {2})) (1 - \tau_ {1} ^ {2} - \tau_ {2} ^ {2} + O (\epsilon^ {3 / 2}))}{1 + O (\epsilon)}} \sigma^ {2} \right. \\ \left. - 2 \tau_ {2} \omega_ {2} \rho \sigma^ {2} - 2 \tau_ {2} \omega_ {1} \nu \rho^ {2} \sigma^ {2} - 2 \tau_ {1} \nu \rho^ {2} \sigma^ {2}\right). \tag {295} \\ \end{array}
$$
We proceed with solving the above optimization program. Taking the derivative of the objective function with respect to  $\eta_{1}$  and  $\eta_{2}$  yields the following:
$$
\frac {\eta_ {2}}{\sqrt {1 - \eta_ {1} ^ {2} - \eta_ {2} ^ {2}}} = \rho \sqrt {1 + \rho^ {2}} + O (\epsilon), \tag {296}
$$
$$
\frac {\eta_ {1}}{\sqrt {1 - \eta_ {1} ^ {2} - \eta_ {2} ^ {2}}} = \rho^ {2} + O (\epsilon). \tag {297}
$$
Solving the above set of equations, we get
$$
\eta_ {1} = \frac {\rho^ {2}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4}}} + O (\epsilon), \tag {298}
$$
$$
\eta_ {2} = \frac {\rho \sqrt {1 + \rho^ {2}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4}}} + O (\epsilon). \tag {299}
$$
Thus, considering the dominant terms, we get the following reconstruction for the third frame
$$
\hat {X} _ {3} ^ {G} = \frac {\rho^ {2}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4}}} \hat {X} _ {1} ^ {G} + \frac {\rho \sqrt {1 + \rho^ {2}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4}}} \hat {X} _ {2} ^ {G} + \frac {\sqrt {2 \epsilon \ln 2}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4}}} X _ {3} + Z _ {3}. \tag {300}
$$
The above reconstruction yields the following distortion for the third frame
$$
\mathbb {E} \left[ \| X _ {3} - \hat {X} _ {3} ^ {G} \| ^ {2} \right] = 2 \left(1 - \sqrt {2 \epsilon \ln 2 \left(1 + \rho^ {2} + 2 \rho^ {4}\right)}\right) \sigma^ {2}. \tag {301}
$$
Finally, consider the reconstruction of the fourth frame as follows
$$
\hat {X} _ {4} ^ {G} = \lambda_ {1} \hat {X} _ {1} ^ {G} + \lambda_ {2} \hat {X} _ {2} ^ {G} + \lambda_ {3} \hat {X} _ {3} ^ {G} + \lambda_ {4} X _ {4} + Z _ {4}, \tag {302}
$$
where  $\hat{X}_4^G\sim \mathcal{N}(0,\sigma^2)$ . The rate constraint of the fourth step implies that
$$
\left(1 - \lambda_ {1} ^ {2} - \lambda_ {2} ^ {2} - \lambda_ {3} ^ {2} + O (\epsilon)\right) (2 \epsilon \ln 2 + O (\epsilon)) = \lambda_ {4} ^ {2} (1 + O (\epsilon)). \tag {303}
$$
$$
\begin{array}{l} \mathbb {E} [ \| X _ {4} - \hat {X} _ {4} ^ {G} \| ^ {2} ] = 2 \sigma^ {2} - 2 \lambda_ {4} \sigma^ {2} - 2 \lambda_ {3} \rho \tau_ {3} \sigma^ {2} - 2 \lambda_ {3} \rho^ {2} \tau_ {2} \omega_ {2} \sigma^ {2} - 2 \lambda_ {3} \rho^ {3} \tau_ {2} \omega_ {1} \nu \sigma^ {2} \\ - 2 \lambda_ {3} \rho^ {3} \tau_ {1} \nu \sigma^ {2} - 2 \lambda_ {2} \rho^ {3} \omega_ {1} \nu \sigma^ {2} - 2 \lambda_ {2} \rho^ {2} \omega_ {2} \sigma^ {2} - 2 \lambda_ {1} \rho^ {3} \nu (304) \\ = 2 \sigma^ {2} - 2 \sqrt {(2 \epsilon \ln 2) (1 - \lambda_ {1} ^ {2} - \lambda_ {2} ^ {2} - \lambda_ {3} ^ {2})} \sigma^ {2} - 2 \lambda_ {3} \rho \tau_ {3} \sigma^ {2} \\ - 2 \lambda_ {3} \rho^ {2} \tau_ {2} \omega_ {2} \sigma^ {2} - 2 \lambda_ {3} \rho^ {3} \tau_ {2} \omega_ {1} \nu \sigma^ {2} - 2 \lambda_ {3} \rho^ {3} \tau_ {1} \nu \sigma^ {2} \\ - 2 \lambda_ {2} \rho^ {3} \omega_ {1} \nu \sigma^ {2} - 2 \lambda_ {2} \rho^ {2} \omega_ {2} \sigma^ {2} - 2 \lambda_ {1} \rho^ {3} \nu + O (\epsilon). (305) \\ \end{array}
$$
We take the derivative of the above expression with respect to  $\lambda_{1},\lambda_{2}$  and  $\lambda_{3}$  and we get
$$
\frac {\lambda_ {1}}{\sqrt {1 - \lambda_ {1} ^ {2} - \lambda_ {2} ^ {2} - \lambda_ {3} ^ {2}}} = \rho^ {3} + O (\epsilon), \tag {306}
$$
$$
\frac {\lambda_ {2}}{\sqrt {1 - \lambda_ {1} ^ {2} - \lambda_ {2} ^ {2} - \lambda_ {3} ^ {2}}} = \rho^ {2} \sqrt {1 + \rho^ {2}} + O (\epsilon), \tag {307}
$$
$$
\frac {\lambda_ {3}}{\sqrt {1 - \lambda_ {1} ^ {2} - \lambda_ {2} ^ {2} - \lambda_ {3} ^ {2}}} = \rho \sqrt {1 + \rho^ {2} + 2 \rho^ {4}} + O (\epsilon). \tag {308}
$$
Solving the above set of equations yields the following
$$
\lambda_ {1} = \frac {\rho^ {3}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} + O (\epsilon), \tag {309}
$$
$$
\lambda_ {2} = \frac {\rho^ {2} \sqrt {1 + \rho^ {2}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} + O (\epsilon), \tag {310}
$$
$$
\lambda_ {3} = \frac {\rho \sqrt {1 + \rho^ {2} + 2 \rho^ {4}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} + O (\epsilon). \tag {311}
$$
Thus, considering the dominant terms, we can write
$$
\begin{array}{l} \hat {X} _ {4} ^ {G} = \frac {\rho^ {3}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} \hat {X} _ {1} ^ {G} + \frac {\rho^ {2} \sqrt {1 + \rho^ {2}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} \hat {X} _ {2} ^ {G} \\ + \frac {\rho \sqrt {1 + \rho^ {2} + 2 \rho^ {4}}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} \hat {X} _ {3} ^ {G} + \frac {\sqrt {2 \epsilon \ln 2}}{\sqrt {1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}}} X _ {4} + Z _ {4}. \tag {312} \\ \end{array}
$$
The distortion term then becomes:
$$
\mathbb {E} \left[ \| X _ {4} - \hat {X} _ {4} ^ {G} \| ^ {2} \right] = 2 \left(1 - \sqrt {2 \epsilon \ln 2 \left(1 + \rho^ {2} + 2 \rho^ {4} + 4 \rho^ {6}\right)}\right) \sigma^ {2}. \tag {313}
$$
Now, we use induction to derive the terms for  $T$  frames. Define
$$
\begin{array}{l} \Delta_ {\mathrm {F M D}, j} := \sqrt {1 + \sum_ {i = 1} ^ {j - 1} 2 ^ {j - 1 - i} \rho^ {2 (j - i)}}, \quad j = 2, \dots , T (314) \\ = \sqrt {1 + \rho^ {2} \frac {\left(2 \rho^ {2}\right) ^ {j - 1} - 1}{2 \rho^ {2} - 1}}. (315) \\ \end{array}
$$
$$
\hat {X} _ {j} ^ {G} = \sum_ {i = 1} ^ {j - 1} \frac {\Delta_ {\mathrm {F M D} , i} \rho^ {j - i}}{\Delta_ {\mathrm {F M D} , j}} \hat {X} _ {i} ^ {G} + \frac {\sqrt {2 \epsilon \ln 2}}{\Delta_ {\mathrm {F M D} , j}} X _ {j} + Z _ {j}, \quad j = 2, \dots , T, \tag {316}
$$
where  $Z_{j}$  is a Gaussian random variable independent of  $(\hat{X}_1^G,\dots ,\hat{X}_{j - 1}^G,X_j)$  and its variance is such that  $\mathbb{E}[(\hat{X}_j^G)^2 ] = \sigma^2$  . The distortion is given by the following expression
$$
D _ {\mathrm {F M D}, j} = \mathbb {E} \left[ \| X _ {j} - \hat {X} _ {j} \| ^ {2} \right] = 2 \left(1 - \Delta_ {\mathrm {F M D}, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \quad j = 2, \dots , T. \tag {317}
$$
For the special case where  $\rho = 1$ , then the distortion simplifies to the following
$$
\mathbb {E} \left[ \| X _ {j} - \hat {X} _ {j} \| ^ {2} \right] = 2 \left(1 - 2 ^ {\frac {j - 1}{2}} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \quad j = 2, \dots , T, \tag {318}
$$
which shows an exponential decrease at each step.
Distortion Analysis for 0-PLF-JD:
In this case, the proof for  $T$  frames is similar to (254)-(261). Thus, we have
$$
\hat {X} _ {j} ^ {G} = \rho \hat {X} _ {j - 1} ^ {G} + \sqrt {\left(1 - \rho^ {2}\right) 2 \epsilon \ln 2} X _ {j} + Z _ {j}, j = 2, \dots , T, \tag {319}
$$
where  $Z_{j}$  is a Gaussian random variable independent of  $(\hat{X}_{j - 1}^{G}, X_{j})$  and its variance is such that  $\mathbb{E}[(\hat{X}_T^G)^2] = \sigma^2$ . It should be mentioned that preserving the correlation coefficients, e.g.,  $\mathbb{E}[\hat{X}_j^G\hat{X}_{j - 1}^G] = \rho$ , needs some correction terms of  $O(\epsilon)$  as discussed in (257). However, as shown in (261), these correction terms do not contribute to dominant terms of distortion and hence, they can be ignored in the presentation of (319). Now, define
$$
\Delta_ {\mathrm {J D}, j} := \rho^ {2 (j - 1)} + \sqrt {1 - \rho^ {2}} \left(\sum_ {i = 0} ^ {j - 2} \rho^ {2 i}\right), \quad j = 2, \dots , T, \tag {320}
$$
and notice that
$$
\begin{array}{l} D _ {\mathrm {J D}, j} := \mathbb {E} \left[ \| X _ {j} - \hat {X} _ {j} \| ^ {2} \right] (321) \\ = 2 \sigma^ {2} - 2 \mathbb {E} \left[ X _ {j} \hat {X} _ {j} \right] (322) \\ = 2 \sigma^ {2} - 2 \mathbb {E} \left[ X _ {j} \left(\rho \hat {X} _ {j - 1} ^ {G} + \sqrt {\left(1 - \rho^ {2}\right) 2 \epsilon \ln 2} X _ {j}\right) \right] (323) \\ = 2 \sigma^ {2} - 2 \mathbb {E} \left[ X _ {j} \left(\rho^ {j - 1} X _ {1} + \sqrt {1 - \rho^ {2}} \left(\rho^ {j - 2} X _ {2} + \dots + X _ {j}\right)\right) \right] \sqrt {2 \epsilon \ln 2} \sigma^ {2} (324) \\ = 2 \left(1 - \Delta_ {\mathrm {J D}, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}. (325) \\ \end{array}
$$
For the special case of  $\rho = 1$ , we get  $\Delta_{\mathrm{JD},j} = 1$  which remains a constant across different steps.
H Universality Statement for Gauss-Markov Source Model
H.1 MMSE Representations for a Given Rate
For a given rate tuple  $R$ , the minimum distortions achievable by MMSE representations are derived in [28, 37] and are given by
$$
D _ {1} ^ {\min } = \sigma_ {1} ^ {2} 2 ^ {- 2 R _ {1}}, \tag {326}
$$
$$
D _ {2} ^ {\min } = \left(\rho_ {1} ^ {2} \frac {\sigma_ {2} ^ {2}}{\sigma_ {1} ^ {2}} D _ {1} ^ {\min } + \sigma_ {N _ {1}} ^ {2}\right) 2 ^ {- 2 R _ {2}}, \tag {327}
$$
$$
D _ {3} ^ {\min } = \left(\rho_ {2} ^ {2} \frac {\sigma_ {3} ^ {2}}{\sigma_ {2} ^ {2}} D _ {2} ^ {\min } + \sigma_ {N _ {2}} ^ {2}\right) 2 ^ {- 2 R _ {3}}, \tag {328}
$$
$$
\sigma_ {N _ {1}} ^ {2} := \left(1 - \rho_ {1} ^ {2}\right) \sigma_ {2} ^ {2}, \tag {329}
$$
$$
\sigma_ {N _ {2}} ^ {2} := \left(1 - \rho_ {2} ^ {2}\right) \sigma_ {3} ^ {2}. \tag {330}
$$
The above distortions are achieved by the following optimal reconstructions  $\hat{\mathbf{X}}_r$  given in [28]. Notice that the MMSE representation is  $\mathsf{X}_r^{\mathrm{RD}} = \hat{\mathsf{X}}_r$ , i.e., the functions  $\eta_{1}(.)$  and  $\eta_{2}(.,.)$  of iRDP region  $\mathcal{C}_{\mathrm{RDP}}$  (Definition 4) are identity functions (this statement follows from Theorem 5). Now, we choose the reconstruction  $\hat{\mathbf{X}}_r$  in the following.
The reconstruction  $\hat{X}_{r,1}$  is chosen such that  $\hat{X}_{r,1} \to X_1 \to (X_2, X_3)$  holds a Markov chain and
$$
X _ {1} = \hat {X} _ {r, 1} + Z _ {1}, \tag {331}
$$
where  $\hat{X}_{r,1} \sim \mathcal{N}(0, \sigma_1^2 - D_1^{\min})$  and  $Z_1 \sim \mathcal{N}(0, D_1^{\min})$  are independent random variables. Then, the reconstruction  $\hat{X}_{r,2}$  is chosen as follows. Let
$$
W _ {2} := \rho_ {1} \frac {\sigma_ {2}}{\sigma_ {1}} Z _ {1} + N _ {1}, \tag {332}
$$
which is the innovation from  $\hat{X}_{r,1}$  to  $X_{2}$ . Now, we find the random variables  $\hat{W}_2$  and  $Z_{2}$  such that