SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark 1 (Encoded Representations): The proof of the inner bound in Theorem 3 in Appendix D provides an operational interpretation to the auxiliary random variables  $\mathsf{X}_r = (X_{r,1}, X_{r,2}, X_{r,3})$  defined in iRDP region in Definition 4. In particular,  $X_{r,j}$  is a lossy version of the source sample  $X_j$  generated by the encoder in step  $j$ . It is compressed and transmitted to the decoder at rate  $R_j$  in (10). We refer to  $\mathsf{X}_r$  as the encoded representation of the source  $\mathsf{X}$ . The Markov chains (13)-(14) indicate that without loss of optimality, an encoded representation  $X_{r,j}$  can be computed from the source  $X_j$  and past reconstructions  $X_{r,1}, \ldots, X_{r,j-1}$  without using past source samples  $X_1, \ldots, X_{j-1}$ .
Remark 2 (Deterministic Reconstructions): Note that the reconstruction functions generating  $\hat{X}$  in Definition 4 are deterministic functions of the encoded representations (c.f. (12)). In particular, the shared randomness  $K$  is not required in the reconstruction functions. However, as the proof of the inner bound of Theorem 3 illustrates, the shared randomness is required in the compression and construction of  $X_{r,j}$ . Moreover, by following the arguments in [28], one can set the reconstruction function of the last frame to be identity. Thus, in Definition 4, we have set  $\hat{X}_3 = X_{r,3}$  in (12) where  $T = 3$ . In the sequel, for  $T$  frames we will set  $\hat{X}_T = X_{r,T}$ .
Remark 3 The result in Theorem 3 also holds for the PLF-FMD. That is, one can replace the PLF in (9) and (11) by  $\xi_j(P_{X_j},P_{\hat{X}_j})$  and get a similar result (see Appendix D for the justification).
4.2 Gauss-Markov Source Model: RDP Region
In this section, we obtain practical insights through the analysis of the special case of first-order Gauss-Markov sources. We assume that  $X_{1}\sim \mathcal{N}(0,\sigma_{1}^{2})$
$$
X _ {2} = \rho_ {1} \frac {\sigma_ {2}}{\sigma_ {1}} X _ {1} + N _ {1}, \quad X _ {3} = \rho_ {2} \frac {\sigma_ {3}}{\sigma_ {2}} X _ {2} + N _ {2}, \tag {16}
$$
where  $N_{j}$  is independent of  $X_{j}$  with mean zero and variance  $(1 - \rho_j^2)\sigma_{j + 1}^2$  for  $j = 1,2$ . Note that the model extends naturally to the case of  $T$  time-steps. The perception metric is assumed to be the Wasserstein-2 distance, i.e.,  $\phi_j(P_{X_1\dots X_j},P_{\hat{X}_1\dots \hat{X}_j})\coloneqq W_2^2 (P_{X_1\dots X_j},P_{\hat{X}_1\dots \hat{X}_j})$ . For the PLF-FMD, the perception metric is given by  $\xi_{j}(P_{X_{j}},P_{\hat{X}_{j}})\coloneqq W_{2}^{2}(P_{X_{j}},P_{\hat{X}_{j}})$ .
The following result states that for Gaussian sources, jointly Gaussian reconstructions are optimal. Thus, for a given tuple  $(\mathsf{D},\mathsf{P})$ , the characterization of  $\mathcal{R}(\mathsf{D},\mathsf{P})$  becomes computable.
Theorem 4 For the Gauss-Markov source model, any tuple  $(\mathsf{R},\mathsf{D},\mathsf{P})\in \mathcal{C}_{\mathsf{RDP}}$  can be attained by a jointly Gaussian distribution over  $X_{r}$  and identity mappings for  $\eta_j(\cdot)$  in Definition 4.
Proof: See Appendix E.
Generally, the optimized distribution in the above theorem may not admit a simple form. In the special case of  $T = 2$  frames, the optimal reconstructions are given in Appendix E. To obtain practical insights, we consider various asymptotic operating regimes and provide a detailed analysis in Appendix F for the case of  $T = 2$  frames and with  $\sigma_1^2 = \sigma_2^2$ . A summary of these results is provided in Table 2 in the same Appendix. We briefly summarize some of these results next.
4.3 Gauss-Markov Source Model: Extremal Rates
One of the key observations of this paper is that the choice of PLF has implication on the rate allocation across different frames. Specifically, first consider the case when both  $R_{1} = R_{2}$  are small i.e.,  $R_{1} = R_{2} = \epsilon$  (for small enough  $\epsilon$ ). We discuss how each PLF affects the reconstruction in the second step. In the first step, we note that reconstruction in both cases must be identical and of the form  $\hat{X}_{1}^{G} = \sqrt{2\epsilon\ln 2} X_{1} + Z_{1}$  where  $Z_{1}\sim \mathcal{N}(0,(1 - 2\epsilon \ln 2)\sigma^{2})$  is independent of  $X_{1}$ ; the resulting distortion is given by  $D_{1} = 2(1 - \sqrt{2\epsilon\ln 2})\sigma^{2}$ . However, the reconstructions in the second steps will be different for the two measures. For simplicity, we consider the extreme case when  $\rho = 1$  (i.e., when  $X_{2} = X_{1}$ ). Here, the PLF-JD metric is required to preserve perfect correlation and thus has to set  $\hat{X}_{2}^{G} = \hat{X}_{1}^{G}$  and results in  $D_{2} = D_{1}$ . In other words, the decoder in the second step is unable to use any information transmitted in the second step as 0-PLF-JD enforces the stringent constraint  $\hat{X}_{2}^{G} = \hat{X}_{1}^{G}$ . In contrast, for the PLF-FMD metric, it can be shown that the reconstruction in the second step for  $\rho = 1$  reduces to  $\hat{X}_{2}^{G} = \sqrt{2}\sqrt{2\epsilon\ln 2} X_{1} + Z_{\mathrm{FMD}}$  and the associated distortion is given by  $D_{2} = 2(1 - \sqrt{4\epsilon\ln 2})\sigma^{2}$ , which is lower than PLF-JD. Extending this example to  $T$  steps (with  $\rho = 1$ ), we note that PLF-JD will always be forced to output  $\hat{X}_{1}$ , while the reconstruction using PLF-FMD will successively improve. The following theorem formalizes this observation.
Theorem 5 For sufficiently small  $\epsilon$ , let  $R_{j} = \epsilon$  and suppose that  $\rho_{j} = \rho$  and  $\sigma_{j} = \sigma$ , for  $j = 1, \ldots, T$ . The achievable distortions  $D_{FMD,j}$  (for 0-PLF-FMD) and  $D_{JD,j}$  (for 0-PLF-JD) are:
$$
D _ {F M D, j} = 2 \left(1 - \Delta_ {F M D, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \quad D _ {J D, j} = 2 \left(1 - \Delta_ {J D, j} \sqrt {2 \epsilon \ln 2}\right) \sigma^ {2}, \tag {17}
$$
where  $\Delta_{FMD,j} := \sqrt{1 + \rho^2\frac{(2\rho^2)^{j-1}-1}{2\rho^2-1}}$  and  $\Delta_{JD,j} := \rho^{2(j-1)} + \mathbb{1}\{j \geq 2\} \cdot \sqrt{1 - \rho^2} (\sum_{i=0}^{j-2} \rho^{2i})$ .
Proof: See Appendix G.
In particular, specializing to  $\rho = 1$ ,  $\Delta_{\mathrm{FMD},j} = 2^{\frac{j - 1}{2}}$  and  $\Delta_{\mathrm{JD},j} = 1$ . This shows that the decrease in  $D_{\mathrm{FMD},j}$  is exponential at each step which implies the ability of decoder based on 0-PLF-FMD in correcting mistakes and not propagating them in future reconstructions. However, as discussed previously the decoder which uses 0-PLF-JD is stuck at  $\hat{X}_j = \hat{X}_1$  when  $\rho = 1$  and results in no subsequent improvement in the distortion. We call this behaviour as permanence of error. This phenomenon is magnified in the case when  $R_1 \to 0$  and  $R_2 \to \infty$ , treated in Table 2 (Appendix F) as the PLF-JD severely constrains the decoder to copy the previous noisy reconstruction while the flexibility provided by PLF-FMD reduces the distortion in the second step.
The case when  $R_{1} \to \infty$  and  $R_{2} = \epsilon$  treated in Table 2 in Appendix F corresponds to the case when  $X_{1}$  is sent at a sufficiently high rate (as is the case with some I-frames) while  $X_{2}$  is sent at a low rate. Naturally, we have  $\hat{X}_{1}^{G} \approx X_{1}$  for both PLFs. On the other hand, we once again see a qualitatively different behaviour in the reconstruction of  $X_{2}$ . For the case of 0-PLF-FMD, we have  $\hat{X}_{2}^{G} \approx (1 - O(\epsilon))\hat{X}_{1}^{G} + O(\epsilon)X_{2}$ , i.e., the decoder essentially copies the previous frame with
little contribution from the second step. In contrast, for the case of 0-PLF-JD, it can be shown that  $\hat{X}_2^G\approx \rho_1\hat{X}_1^G +O(\sqrt{\epsilon})X_2 + Z_{\mathrm{JD}}$  , where  $Z_{\mathrm{JD}}$  is independent Gaussian noise with variance close to  $1 - \rho^2$  . We note that the PLF-JD metric prevents the decoder from simply "copying" the previous frame, but instead forces the decoder to generate a more diverse representation consistent with the joint distribution between the two frames.
4.4 Universal Representations for Gauss-Markov Source Model
In this section, we show that the Gauss-Markov source model admits universal encoded representations. Such representations can be transformed through appropriate reconstruction functions to achieve the entire DP rate region. This is the counterpart of the result for general sources in Theorem 1 where it is shown that the MMSE reconstructions can be transformed to some target reconstructions satisfying the 0-PLF-FMD with at most a factor-2 increase in distortion. In contrast, we demonstrate that the Gauss-Markov model admits exact universality i.e., target reconstructions proposed in this section achieve all points in the iDP rate region. Interestingly, the transformation is linear with possibly some additive noise. First, we formalize the notion of universal representations.
Definition 5 (iDP-Tradeoff) For a given rate tuple  $\mathsf{R}$ , the optimal iDP-tradeoff is the closure of the set of all tuples  $(\mathsf{D},\mathsf{P})$  such that  $(\mathsf{R},\mathsf{D},\mathsf{P})\in \mathcal{C}_{\mathsf{RDP}}$  and is denoted by  $\mathcal{DP}(\mathsf{R})$ .
Definition 6 (Universal Representation) A given encoded representation  $\mathsf{X}_r$  is called universal with respect to rate tuple  $\mathsf{R}$  if it satisfies the rate constraints (10) and the Markov chains in (13)-(14) and for each  $(\mathsf{D},\mathsf{P})\in \mathcal{DP}(\mathsf{R})$ , there exists a reconstruction  $\hat{\mathsf{X}}$  generated from  $P_{\hat{\mathsf{X}}|\mathsf{X}_r}$  achieving it.
For the Gauss-Markov source model, we show that the MMSE reconstruction admits a universal representation. We consider the reconstruction  $\hat{\mathbf{X}}_r$  that achieves minimum distortion in the  $\mathcal{DP}(\mathsf{R})$  region. This point is explicitly characterized in Appendix H.1. Furthermore, following Theorem 4, since the reconstruction functions  $\eta_j(\cdot)$  are identity, the MMSE reconstruction is equivalent to MMSE representation i.e.,  $\hat{\mathbf{X}}_r = \mathbf{X}_r^{\mathrm{RD}}$ . The following theorem establishes that any point in  $\mathcal{DP}(\mathsf{R})$  can be achieved from  $\hat{\mathbf{X}}_r$ .
Theorem 6 For the Gauss-Markov source model and a given rate tuple  $\mathsf{R}$  with strictly positive components, let the MMSE representation be denoted as  $X_{r}^{RD} = (X_{r,1}^{RD}, X_{r,2}^{RD}, X_{r,3}^{RD})$ . Let  $(\mathsf{D}, \mathsf{P}) \in \mathcal{DP}(\mathsf{R})$  and let  $\hat{\mathsf{X}} = (\hat{X}_{1}, \hat{X}_{2}, \hat{X}_{3})$  be the corresponding reconstruction achieving it. Then there exist  $\kappa_{1}, \theta_{1}, \theta_{2}, \psi_{1}, \psi_{2}$  and  $\psi_{3}$  and noise variables  $(Z_{1}, Z_{2}, Z_{3})$  independent of  $(X_{r,1}^{RD}, X_{r,2}^{RD}, X_{r,3}^{RD})$ , which satisfy the following
$$
\hat {X} _ {1} = \kappa_ {1} X _ {r, 1} ^ {R D} + Z _ {1}, \quad \hat {X} _ {2} = \theta_ {1} X _ {r, 1} ^ {R D} + \theta_ {2} X _ {r, 2} ^ {R D} + Z _ {2}, \quad \hat {X} _ {3} = \psi_ {1} X _ {r, 1} ^ {R D} + \psi_ {2} X _ {r, 2} ^ {R D} + \psi_ {3} \hat {X} _ {r, 3} ^ {R D} + Z _ {3}.
$$
Proof: See Appendix H.2.
The above theorem indicates that the MMSE representation can be linearly transformed to achieve any point in  $\mathcal{D}\mathcal{P}(\mathsf{R})$ . In general the MMSE representation may have to be degraded through additional noise terms. In the proof of Theorem 6 we identify conditions when such degradation is not needed.
As discussed in Appendix H.2, Theorem 6 holds for both PLFs. This suggests the idea that one can train an encoder to get MMSE representations which are oblivious to the choice of PLF. Then, the decoder can generate a reconstruction which satisfies either of PLFs by simply applying a linear transformation to the MMSE representation. Thus, the task of choosing the right PLF can be assigned to the decoder based on distortion and perception requirements. We conclude by noting that Appendix H.3 provides an example where the coefficients in Theorem 6 can be computed explicitly.
We conduct experiments on the MovingMNIST dataset [29] (with 1 digit) using Wasserstein GAN [30], to verify the implications of our theoretical claims to perceptual video compression. Additional results on the KTH dataset [31] are available in Appendix J.3. Our compression network is built on the scale-space flow model [32] and conditional module [33]. For a given rate and PLF, we obtain different distortion-perception tradeoff points by optimizing the weighted sum between distortion
(a) Ground-truth GOP and their optimal reconstructions with different PLFs for  $R_{1} = R_{2} = R_{3} = 12$  bits.
(b) Distortion per frame  $(X_{i} - \hat{X}_{i})^{2}$  for different rates with  $i = 1,2,3$
Figure 2: Permanence of Error Phenomenon. In (a), we visually compare the reconstructions. Note that  $\hat{X}_1$  is the same for both 0-PLF-JD and 0-PLF-PMD. In (b), we show the framewise distortion for different  $(R_2, R_3)$ .
and perception losses. Details about the architecture and training procedure are available in the Appendix J.1. The experimental setup is focused on validating our theory, rather than proposing state-of-the-art neural network architectures. Accordingly, we begin by (1) validating Theorems 1 and 2, which characterize the factor-of-two bounds on the distortion of 0-PLF reconstructions (2) empirically demonstrating the error permanence phenomenon of the PLF-JD in Section 4.3 and (3) computing the DP tradeoff function experimentally as well as confirming that the MMSE reconstruction provides near universal representations, as motivated by the results in Section 4.4.
As our first experimental result in Table 1, we validate the factor of two bounds in Theorems 1 and 2. We consider the compression of two frames  $X_{1}$  and  $X_{2}$  at rates  $R_{1}$  and  $R_{2}$  respectively. The compression of  $X_{1}$  is performed without any prior reference and corresponds to the compression of the "I-frame", while the compression of  $X_{2}$  corresponds to the "P-frame", using  $X_{1}$  as the reference. We consider the cases when either  $R_{1} = \infty$  or  $R_{1} = 12$  bits, where the former corresponds to lossless compression of  $X_{1}$  and the latter corresponds to the low rate regime (see Appendix I for a justification). The average distortion for the
Table 1: Distortions of optimal reconstructions at different regime (  $\checkmark$  means factor of 2 holds and  $\times$  means otherwise). Distortion is scaled by  ${10}^{-2}$  .
R2	MMSE	0-PLF-FMD	0-PLF-JD
1	1.08 ± 0.01	1.74 ± 0.02√	2.05 ± 0.03√
2	0.88 ± 0.01	1.39 ± 0.03√	1.46 ± 0.02√
3.17	0.53 ± 0.01	0.76 ± 0.01√	0.79 ± 0.01√
(a) Case 1:  $R_{1} = \infty$  bits
R2	MMSE	0-PLF-FMD	0-PLF-JD
4	1.23 ± 0.01	2.21 ± 0.04√	2.36 ± 0.04 √
8	1.04 ± 0.01	1.78 ± 0.03 √	2.28 ± 0.03 X
12	0.89 ± 0.02	1.43 ± 0.02 √	2.26 ± 0.03 X
∞	0.0	0.0 √	2.18 ± 0.02 X
(b) Case 2:  $R_{1} = 12$  bits  $(\epsilon)$ .
first frame when  $R_{1} = 12$  is 0.0124 for the MMSE reconstruction and 0.0235 for the 0-PLF reconstruction, thus satisfying the factor of two bound. In compression of  $X_{2}$ , we systematically vary the value of the rate  $R_{2} \in \{4,8,12,\infty\}$ . Following Table 1b, for 0-PLF-JD reconstruction, only  $R_{2} = 4$  bits (low rate) satisfies the factor of two bounds as expected. Intuitively, even as more bits are acquired, the 0-PLF-JD criteria actively restricts improving the reconstructions, resulting in persistently higher distortion. Even when  $R_{2} = \infty$ , the distortion remains non-zero as the decoder is forced to maintain temporal consistency with  $\hat{X}_{1}$ . In contrast, for FMD, the factor of 2 bound holds at all rates, consistent with Theorem 1.
In Fig. 2, we present our experimental results with a group of pictures (GOP) of size 3 (i.e. one I-frame followed by two P-frames). In Fig. 2a, we visualize sample reconstructions for MSE, 0-PLF-FMD and 0-PLF-JD cases when operating in the low-rate regime with  $R_{j} = 12$  bits for  $j = 1,2,3$ . Note that given an incorrect digit reconstruction in  $\hat{X}_1$ , the decoder with 0-PLF-JD consistently produces incorrect digits (or content) while the 0-PLF-FMD gradually "corrects" it, which confirms the error permanence phenomenon discussed in the theoretical analysis in Section 4.3 and Table 2 in Appendix F. We also plot the framewise distortion in Fig. 2b to show the difference in achievable distortion two perception metrics across different values for  $R_2$  and  $R_3$  as a function of the frame index. Consistent with Theorem 5, the achievable distortion decreases much faster for 0-PLF-FMD than 0-PLF-JD for all selection of rates. Finally, we show similar results in Figure 3 for UVG dataset.
In Fig. 4, we plot the tradeoff curves between distortion and perception for the second reconstruction  $\hat{X}_2$  for both optimal (end-to-end) and universal representations for two cases: when  $R_1 = \infty$  and  $R_1 = 12$  bits and for a range of values for  $R_2$ . In general, the curves for both universal and optimal
Figure 3: Error permanence phenomenon on the UVG dataset. The PLF-JD reconstructions propagate the flaws in the color tone from the previous I-frame reconstruction while the PLF-FMD is able to fix these flaws. Compression rate for I-frame and P-frame are 0.144bpp (low rate) and 4.632bpp (high rate) respectively.
representations are relatively close to each other at all rate regimes. The general shape of every curve is relatively similar with the exception of the PLF-JD metric in Fig. 4b, where the curves for different rates seemingly converge since increasing the rate does not significantly improve the distortion in this case as noted previously. Finally, as the universal encoders are derived from MMSE solutions, these results imply that one can simply send the MMSE representation to the decoder and the user can flexibly change the DP tradeoff up to their requirements. We further note that even when the end-to-end model targets an operating point different from the MMSE reconstruction, the latter is still required to estimate the motion flow vectors best. The universal representation provides a natural way to reconstruct the MMSE reconstruction from the encoder output. In the plots of Fig. 4, we leverage on established universality results for I-frame compression in prior works [16] to construct the MMSE representation for motion compensation as we have a GOP of size 2.
(a) Case 1:  $R_{1} = \infty$ ,  $R_{2} = \{1,2,3.17\}$  bits
(b) Case 2:  $R_{1} = 12$ ,  $R_{2} = \{4,8,12,\infty\}$  bits
Figure 4: RDP tradeoff curves for end-to-end and universal models. We plot the tradeoff for the two regimes:  $R_{1} = \infty$  and  $R_{1} = \epsilon$  in (a) and (b) respectively. The universal and optimal curves are close to each other.
Finally in Appendix J.5, we consider the ability of the decoder to generate diverse reconstructions when operating under either PLF-JD or PLF-FMD. We focus on the case when  $X_{1}$  is transmitted losslessly and when  $X_{2}$  is compressed at low rates. Consistent with the theoretical analysis in Section 4.2 and Table 2 in Appendix F, the decoder optimized for PLF-JD is capable of producing diverse reconstructions by mimicking the actual motion between the frames. The PLF-FMD leads to reconstructions that are highly correlated and less desirable.
This work examines different perception loss functions for causal video coding, establishing its key theoretical properties such as the operational RDP region and universality principle. Our analysis highlights that while 0-PLF-JD reconstruction preserves temporal correlation, it is susceptible to the error permanence phenomenon. Moreover, our investigation of universality reveals that the encoder can transform the MMSE representation to other points on the DP tradeoffs, irrespective of the PLF. We suggest future research directions such as exploring region-based perceptual metrics [34], incorporating image-aware bits allocation, and leveraging conditional perception metric [18].
[42] S. Hong, D. Yang, Y. Jang, T. Zhao, and H. Lee, "Diversity-sensitive conditional generative adversarial networks," in 7th International Conference on Learning Representations, ICLR 2019. International Conference on Learning Representations, ICLR, 2019.
[43] R. Yang and S. Mandt, "Lossy image compression with conditional diffusion models," arXiv preprint arXiv:2209.06950, 2022.
A Distortion Analysis for 0-PLF-FMD
Recall the definition of Wasserstein-2 distance [26] as follows. For given distributions  $P_{X_j}$  and  $P_{\tilde{X}_j}$ , let
$$
W _ {2} ^ {2} \left(P _ {\tilde {X} _ {j}}, P _ {X _ {j}}\right) := \inf  \mathbb {E} [ \| X _ {j} - \tilde {X} _ {j} \| ^ {2} ], \tag {18}
$$
where the infimum is over all joint distributions of  $(X_{j},\tilde{X}_{j})$  with marginals  $P_{X_j}$  and  $P_{\tilde{X}_j}$ .
$$
\Phi_ {\mathbb {D} ^ {0}} \left(P _ {\mathrm {M} | \mathrm {X} K}\right) = \{\mathbb {D}: D _ {j} \geq \mathbb {E} _ {P} [ \| X _ {j} - \tilde {X} _ {j} \| ^ {2} ] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {j}}, P _ {X _ {j}}\right), j = 1, 2, 3 \}, \tag {19}
$$
Furthermore, we also have that:
$$
\Phi_ {\mathsf {D} ^ {0}} \left(P _ {\mathsf {M} | \times K}\right) \supseteq \{\mathsf {D}: D _ {j} \geq 2 \mathbb {E} _ {P} [ \| X _ {j} - \tilde {X} _ {j} \| ^ {2} ], \quad j = 1, 2, 3 \}, \tag {20}
$$
Proof: Define
$$
\mathcal {D} ^ {0} := \left\{\mathrm {D}: D _ {j} \geq \mathbb {E} \left[ \left\| X _ {j} - \tilde {X} _ {j} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {j}}, P _ {X _ {j}}\right), j = 1, 2, 3 \right\}. \tag {21}
$$
First, we show that  $\Phi_{\mathsf{D}^0}(P_{\mathsf{M}|\mathsf{X}K}) \subseteq \mathcal{D}^0$ . For any  $\mathsf{D} \in \Phi_{\mathsf{D}^0}(P_{\mathsf{M}|\mathsf{X}K})$ , there exists  $\hat{X}_{\mathsf{D}^0} = (\hat{X}_{D_1^0}, \hat{X}_{D_2^0}, \hat{X}_{D_3^0})$  jointly distributed with  $(\mathsf{M}, \mathsf{X}, K)$  such that
$$
\mathbb {E} \left[ \| X _ {j} - \hat {X} _ {D _ {0} ^ {0}} \| ^ {2} \right] \leq D _ {j}, \quad j = 1, 2, 3, \tag {22}
$$
$$
P _ {X _ {j}} = P _ {\hat {X} _ {D _ {j} ^ {0}}}. \tag {23}
$$
Then, for example, the analysis for the second frame is as follows
$$
\begin{array}{l} D _ {2} \geq \mathbb {E} \left[ \left\| X _ {2} - \hat {X} _ {D _ {2} ^ {0}} \right\| ^ {2} \right] (24) \\ = \mathbb {E} \left[ \left\| \left(X _ {2} - \tilde {X} _ {2}\right) - \left(\hat {X} _ {D _ {2} ^ {0}} - \tilde {X} _ {2}\right) \right\| ^ {2} \right] (25) \\ = \mathbb {E} \left[ \left\| X _ {2} - \tilde {X} _ {2} \right\| ^ {2} \right] + \mathbb {E} \left[ \left\| \tilde {X} _ {2} - \hat {X} _ {D _ {2} ^ {0}} \right\| ^ {2} \right] (26) \\ \geq \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {\hat {X} _ {D _ {2} ^ {0}}}\right) (27) \\ = \mathbb {E} \left[ \left\| X _ {2} - \tilde {X} _ {2} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {X _ {2}}\right), (28) \\ \end{array}
$$
where (26) holds because both  $\tilde{X}_2$  and  $\hat{X}_{D_2^0}$  are functions of  $(M_1,M_2,K)$  and thus the MMSE  $(X_{2} - \tilde{X}_{2})$  is uncorrelated with  $(\hat{X}_{D_2^0} - \tilde{X}_2)$ ; (28) follows because the 0-PLF-FMD implies that  $P_{\hat{X}_{D_2^0}} = P_{X_2}$ . Following similar steps for other frames, we get  $\Phi_{\mathsf{D}^0}(P_{\mathsf{M}|\mathsf{X}K})\subseteq \mathcal{D}^0$ .
Next, we show that  $\mathcal{D}^0\subseteq \Phi_{\mathsf{D}^0}(P_{\mathsf{M}|\times K})$ . Assume that  $\mathsf{D}\in \mathcal{D}^0$ . Let  $\hat{X}_1^*$  be an auxiliary random variable jointly distributed with  $(M_1,K)$  such that it satisfies the following conditions
$$
P _ {\hat {X} _ {1} ^ {*}} = P _ {X _ {1}}, \tag {29}
$$
$$
P _ {\tilde {X} _ {1} \hat {X} _ {1} ^ {*}} = \arg \inf  _ {\bar {P} _ {\tilde {X} _ {1} \hat {X} _ {1} ^ {*}}:} \mathbb {E} _ {\bar {P}} [ \| \tilde {X} _ {1} - \hat {X} _ {1} ^ {*} \| ^ {2} ]. \tag {30}
$$
$$
\bar {P} _ {\tilde {X} _ {1}} = P _ {\tilde {X} _ {1}}
$$
$$
\bar {P} _ {\hat {X} _ {1} ^ {*}} = P _ {\hat {X} _ {1} ^ {*}}
$$
Moreover, let  $\hat{X}_2^*$  be an auxiliary random variable jointly distributed with  $(M_1,M_2,K)$  such that the following two conditions are satisfied