SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
P _ {\hat {X} _ {2} ^ {*}} = P _ {X _ {2}}, \tag {31}
$$
$$
P _ {\tilde {X} _ {2} \hat {X} _ {2} ^ {*}} = \arg \inf  _ {\bar {P} _ {\tilde {X} _ {2} \hat {X} _ {2} ^ {*}}:} \mathbb {E} _ {\bar {P}} [ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} ]. \tag {32}
$$
$$
\bar {P} _ {\tilde {X} _ {2}} = P _ {\tilde {X} _ {2}}
$$
$$
P _ {\hat {X} _ {2} ^ {*}} = P _ {\hat {X} _ {2} ^ {*}}
$$
Similarly, we define  $\hat{X}_3^*$ . Now, notice that since  $\mathsf{D} \in \mathcal{D}^0$ , we have:
$$
D _ {2} \geq \mathbb {E} \left[ \left\| X _ {2} - \tilde {X} _ {2} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {X _ {2}}\right). \tag {33}
$$
It then directly follows that
$$
\mathbb {E} \left[ \| X _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] = \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] + \mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] \tag {34}
$$
$$
= \mathbb {E} \left[ \left\| X _ {2} - \tilde {X} _ {2} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {\hat {X} _ {2} ^ {*}}\right) \tag {35}
$$
$$
= \mathbb {E} \left[ \left\| X _ {2} - \tilde {X} _ {2} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {X _ {2}}\right) \tag {36}
$$
$$
\leq D _ {2}, \tag {37}
$$
(34) follows because  $\tilde{X}_2$  and  $\hat{X}_2^*$  are functions of  $(M_1, M_2, K)$  and thus the MMSE  $(X_2 - \tilde{X}_2)$  is uncorrelated with  $(\hat{X}_2^* - \tilde{X}_2)$ ;
(35) follows from (32);
(36) follows because  $P_{\hat{X}_2^*} = P_{X_2}$ .
Following similar steps for other frames, we get  $\mathsf{D} \in \Phi_{\mathsf{D}^0}(P_{\mathsf{X}_{\mathrm{r}}|\mathsf{X}})$ .
Now, notice that  $W_2^2 (P_{\tilde{X}_2},P_{X_2})\leq \mathbb{E}[\| X_2 - \tilde{X}_2\| ^2 ]$  since the Wasserstein-2 distance takes the infimum over all possible joint distributions  $(X_{2},\tilde{X}_{2})$ , but the expectation in  $\mathbb{E}[\| X_2 - \tilde{X}_2\| ^2 ]$  is taken over the given  $P_{X_2\tilde{X}_2}$ . Thus, we get
$$
\mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2}}, P _ {X _ {2}}\right) \leq 2 \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right]. \tag {38}
$$
B Distortion Analysis for 0-PLF-JD
Let  $\hat{X}_1^*$  be defined as in (29)-(30). Moreover, let  $\hat{X}_2^*$  be an auxiliary random variable jointly distributed with  $(M_1,M_2,K)$  such that the following conditions are satisfied
$$
P _ {\hat {X} _ {2} ^ {*} \mid \hat {X} _ {1} ^ {*} = x _ {1}} = P _ {X _ {2} \mid X _ {1} = x _ {1}}, \quad \forall x _ {1} \in \mathcal {X} _ {1}, \tag {39}
$$
$$
P _ {\tilde {X} _ {2} \hat {X} _ {2} ^ {*} | \hat {X} _ {1} ^ {*} = x _ {1}} = \arg \inf  _ {\tilde {P} _ {\hat {X} _ {2} \hat {X} _ {2} ^ {*} | \hat {X} _ {1} ^ {*} = x _ {1}}:} \mathbb {E} _ {\bar {P}} [ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} | \hat {X} _ {1} ^ {*} = x _ {1} ], \quad \forall x _ {1} \in \mathcal {X} _ {1}. \tag {40}
$$
$$
\bar {P} _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}} = P _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}}
$$
$$
\bar {P} _ {\hat {X} _ {2} ^ {*} | \hat {X} _ {1} ^ {*} = x _ {1}} = P _ {\hat {X} _ {2} ^ {*} | \hat {X} _ {1} ^ {*} = x _ {1}}
$$
Then, the following result holds.
Theorem 2 We have
$$
\Phi_ {\mathsf {D} ^ {0}} ^ {j o i n t} \left(P _ {\mathsf {M} | \mathsf {X} K}\right) \supseteq \{\mathsf {D}: D _ {1} \geq \mathbb {E} \left[ \left\| X _ {1} - \tilde {X} _ {1} \right\| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {1}}, P _ {X _ {1}}\right),
$$
$$
D _ {2} \geq \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {X _ {1}} (x _ {1}) W _ {2} ^ {2} (P _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}}, P _ {X _ {2} | X _ {1} = x _ {1}}),
$$
$$
D _ {3} \geq \mathbb {E} [ \| X _ {3} - \tilde {X} _ {3} \| ^ {2} ] + \sum_ {x _ {1}, x _ {2}} P _ {X _ {1} X _ {2}} (x _ {1}, x _ {2}) W _ {2} ^ {2} \left(P _ {\tilde {X} _ {3} | \hat {X} _ {1} ^ {*} = x _ {1}, \hat {X} _ {2} ^ {*} = x _ {2}}, P _ {X _ {3} | X _ {1} = x _ {1}, X _ {2} = x _ {2}}\right) \}. \tag {41}
$$
$$
\mathcal {D} _ {\mathrm {j o i n t}} ^ {0} := \left\{\mathsf {D}: D _ {1} \geq \mathbb {E} [ \| X _ {1} - \tilde {X} _ {1} \| ^ {2} ] + W _ {2} ^ {2} (P _ {\tilde {X} _ {1}}, P _ {X _ {1}}), \right.
$$
$$
D _ {3} \geq \mathbb {E} [ \| X _ {3} - \tilde {X} _ {3} \| ^ {2} ] + \sum_ {x _ {1}, x _ {2}} P _ {X _ {1} X _ {2}} (x _ {1}, x _ {2}) W _ {2} ^ {2} \left(P _ {\tilde {X} _ {3} | \hat {X} _ {1} ^ {*} = x _ {1}, \hat {X} _ {2} ^ {*} = x _ {2}}, P _ {X _ {3} | X _ {1} = x _ {1}, X _ {2} = x _ {2}}\right) \}. \tag {42}
$$
Now, assume that  $\mathsf{D} \in \mathcal{D}_{\mathrm{joint}}^0$ . For the first frame, recall that  $\hat{X}_1^*$  is an auxiliary random variable jointly distributed with  $(M_1, K)$  such that it satisfies (29)-(30). From similar steps to (34)-(36), it then follows that
$$
\begin{array}{l} \mathbb {E} \left[ \| X _ {1} - \hat {X} _ {1} ^ {*} \| ^ {2} \right] = \mathbb {E} \left[ \| X _ {1} - \tilde {X} _ {1} \| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {1}}, P _ {X _ {1}}\right) (43) \\ \leq D _ {1}. (44) \\ \end{array}
$$
For the second frame, since  $\mathsf{D} \in \mathcal{D}_{\mathrm{joint}}^0$ , we have:
$$
D _ {2} \geq \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {X _ {1}} (x _ {1}) W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2} | X _ {1} = x _ {1}}, P _ {X _ {2} | X _ {1} = x _ {1}}\right). \tag {45}
$$
Recall that  $\hat{X}_2^*$  is an auxiliary random variable jointly distributed with  $(M_1, M_2, K)$  such that (39)-(40) hold. It then directly follows that
$$
\begin{array}{l} \mathbb {E} \left[ \| X _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] = \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] + \mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] (46) \\ = \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} (x _ {1}) \mathbb {E} [ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} | \hat {X} _ {1} ^ {*} = x _ {1} ] (47) \\ = \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} (x _ {1}) W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}}, P _ {\hat {X} _ {2} ^ {*} | \hat {X} _ {1} ^ {*} = x _ {1}}\right) (48) \\ = \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {X _ {1}} (x _ {1}) W _ {2} ^ {2} \left(P _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}}, P _ {X _ {2} | X _ {1} = x _ {1}}\right), (49) \\ \end{array}
$$
(46) follows because  $\tilde{X}_2$  and  $\hat{X}_2^*$  are functions of  $(M_1, M_2, K)$  and thus the MMSE  $(X_2 - \tilde{X}_2)$  is uncorrelated with  $(\hat{X}_2^* - \tilde{X}_2)$ ,
(48) follows from (40),
(49) follows because  $P_{\hat{X}_1^* \hat{X}_2^*} = P_{X_1 X_2}$ .
Following similar steps for the third frame, we get  $\mathsf{D} \in \Phi_{\mathsf{D}^0}(P_{\mathsf{M}|\mathsf{X}K})$ . This concludes the proof.
B.1 A Counterexample for Factor-Two Bound in Case of 0-PLF-JD
Assume that we have only two frames, i.e.,  $D_{3} \to \infty$ . Let  $M_{1}$  be independent of  $X_{1}$  and  $M_{2} = X_{2}$ . Then, we have  $\tilde{X}_{1} = \emptyset$  and  $\tilde{X}_{2} = X_{2}$ . Consider the achievable distortion region of Theorem 2. The distortion of the first step is given by the following
$$
\mathbb {E} \left[ \| X _ {1} - \tilde {X} _ {1} \| ^ {2} \right] + W _ {2} ^ {2} \left(P _ {\tilde {X} _ {1}}, P _ {X _ {1}}\right) = 2 \mathbb {E} \left[ X _ {1} ^ {2} \right]. \tag {50}
$$
For the second frame, we have
$$
\begin{array}{l} \mathbb {E} [ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} ] + \sum_ {x _ {1}} P _ {X _ {1}} (x _ {1}) W _ {2} ^ {2} (P _ {\tilde {X} _ {2} | \hat {X} _ {1} ^ {*} = x _ {1}}, P _ {X _ {2} | X _ {1} = x _ {1}}) \\ = \sum_ {x _ {1}} P _ {X _ {1}} \left(x _ {1}\right) W _ {2} ^ {2} \left(P _ {X _ {2} \mid \hat {X} _ {1} ^ {*} = x _ {1}}, P _ {X _ {2} \mid X _ {1} = x _ {1}}\right) (51) \\ = \sum_ {x _ {1}} P _ {X _ {1}} \left(x _ {1}\right) W _ {2} ^ {2} \left(P _ {X _ {2}}, P _ {X _ {2} \mid X _ {1} = x _ {1}}\right), (52) \\ \end{array}
$$
where (51) follows because  $\tilde{X}_2 = X_2$  and (52) follows because  $X_{2}$  is independent of  $\hat{X}_1^*$  ( $M_1$  is independent of  $X_{1}$ , then  $\hat{X}_1^*$ , which is a function of  $(M_1, K)$ , would be independent of  $X_{1}$  and hence independent of  $X_{2}$ ).
Now, notice that the MMSE distortion of the second step is zero since  $\tilde{X}_2 = X_2$ . However, the achievable distortion of the second step for the reconstruction satisfying 0-PLF JD is given in (52) which clearly does not satisfy the factor-two bound.
C Fixed Encoders Operating at Low rate regime
We consider the class of noisy encoders where the encoder distribution can be written as follows
$$
P _ {X _ {j} \mid M _ {1} \dots M _ {j} K} ^ {\text {n o i s y}} = (1 - \mu) P _ {X _ {j}} + \mu Q _ {X _ {j} \mid M _ {1} \dots M _ {j} K} ^ {\text {n o i s y}}, \quad j = 1, 2, 3. \tag {53}
$$
where  $\mu$  is a sufficiently small constant and the distribution  $Q^{\mathrm{noisy}}(\cdot)$  could be arbitrary conditional distribution with same marginal as  $P_{X_j}$ .
Theorem 3 For the class of encoders given by (53), we have
$$
\Phi_ {\mathsf {D} ^ {0}} ^ {\text {j o i n t}} \left(P _ {\mathsf {M} | \mathsf {X} K} ^ {\text {n o i s y}}\right) \supseteq \{\mathsf {D}: D _ {j} \geq 2 \mathbb {E} _ {P ^ {\text {n o i s y}}} \left[ \| X _ {j} - \tilde {X} _ {j} \| ^ {2} \right] + O (\mu), \quad j = 2, \dots , 3 \}. \tag {54}
$$
Proof: We analyze the distortion for the second frame. A similar argument holds for other frames.
Denote the reconstruction of the second step by  $\hat{X}_2^*$  and consider the expected distortion. From a similar justification starting from (24) and leading to (26), we can write the distortion as follows
$$
\mathbb {E} \left[ \| X _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] = \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] + \mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right]. \tag {55}
$$
Now, we study the expected term  $\mathbb{E}[\| \tilde{X}_2 - \hat{X}_2^*\|^2]$  as follows
$$
\mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] = \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} \left(x _ {1}\right) \mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \mid \hat {X} _ {1} ^ {*} = x _ {1} \right]. \tag {56}
$$
In order to analyze the above expression, we first approximate the MMSE reconstruction  $\tilde{X}_2$  as follows
$$
\begin{array}{l} \tilde {X} _ {2} = \mathbb {E} _ {P ^ {\text {n o i s y}}} [ X _ {2} | M _ {1}, M _ {2}, K ] (57) \\ = (1 - \mu) \mathbb {E} _ {P} [ X _ {2} ] + \mu \mathbb {E} _ {Q ^ {\text {n o i s y}}} [ X _ {2} | M _ {1}, M _ {2}, K ] (58) \\ = \mathbb {E} \left[ X _ {2} \right] + O (\mu), (59) \\ \end{array}
$$
where (58) follows from (53). Moreover, notice that (59) implies that
$$
\begin{array}{l} \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] = \mathbb {E} \left[ \| X _ {2} - \mathbb {E} \left[ X _ {2} \right] + \mu \left(\mathbb {E} _ {Q ^ {\text {n o i s y}}} \left[ X _ {2} \mid M _ {1}, M _ {2}, K \right] - \mathbb {E} [ X _ {2} ]\right) \| ^ {2} \right] (60) \\ = \mathbb {E} [ \| X _ {2} - \mathbb {E} [ X _ {2} ] \| ^ {2} ] + O (\mu). (61) \\ \end{array}
$$
Next, consider the expected term in (56) as follows
$$
\begin{array}{l} \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} \left(x _ {1}\right) \mathbb {E} \left[ \| \tilde {X} _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \mid \hat {X} _ {1} ^ {*} = x _ {1} \right] = \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} \left(x _ {1}\right) \mathbb {E} \left[ \| \mathbb {E} \left[ X _ {2} \right] - \hat {X} _ {2} ^ {*} \| ^ {2} \mid \hat {X} _ {1} ^ {*} = x _ {1} \right] + O (\mu) (62) \\ = \sum_ {x _ {1}} P _ {\hat {X} _ {1} ^ {*}} \left(x _ {1}\right) \mathbb {E} \left[ \| \mathbb {E} \left[ X _ {2} \right] - X _ {2} \| ^ {2} \mid X _ {1} = x _ {1} \right] + O (\mu) (63) \\ = \sum_ {x _ {1}} P _ {X _ {1}} \left(x _ {1}\right) \mathbb {E} \left[ \| \mathbb {E} \left[ X _ {2} \right] - X _ {2} \| ^ {2} \mid X _ {1} = x _ {1} \right] + O (\mu) (64) \\ = \mathbb {E} [ \| \mathbb {E} [ X _ {2} ] - X _ {2} \| ^ {2} ] + O (\mu) (65) \\ = \mathbb {E} \left[ \| \tilde {X} _ {2} - X _ {2} \| ^ {2} \right] + O (\mu), (66) \\ \end{array}
$$
(62) follows from (59);
(63) follows because the 0-PLF-JD implies that  $P_{\hat{X}_2^*|\hat{X}_1^*} = P_{X_2|X_1}$  and  $\mathbb{E}[X_2]$  is just a constant;
(64) follows from 0-PLF-JD where  $P_{\hat{X}_1^*} = P_{X_1}$
(66) follows from (61).
Considering (55) and (66), we get
$$
\mathbb {E} \left[ \| X _ {2} - \hat {X} _ {2} ^ {*} \| ^ {2} \right] = 2 \mathbb {E} \left[ \| X _ {2} - \tilde {X} _ {2} \| ^ {2} \right] + O (\mu). \tag {67}
$$
The proof for the third frame follows similar steps.
Figure 5: Encoded representations and reconstructions of the iRDP region  $C_{\mathrm{RDP}}$ .
D Operational RDP Region
Recall the definition of iRDP region  $\mathcal{C}_{\mathrm{RDP}}$  for first-order Markov sources (Definition 4) as follows. It is the set of all tuples  $(\mathsf{R},\mathsf{D},\mathsf{P})$  satisfying
$$
R _ {1} \geq I \left(X _ {1}; X _ {r, 1}\right), \tag {68}
$$
$$
R _ {2} \geq I \left(X _ {2}; X _ {r, 2} \mid X _ {r, 1}\right), \tag {69}
$$
$$
R _ {3} \geq I \left(X _ {3}; X _ {r, 3} \mid X _ {r, 1}, X _ {r, 2}\right), \tag {70}
$$
$$
D _ {j} \geq \mathbb {E} [ \| X _ {j} - \hat {X} _ {j} \| ^ {2} ], \quad j = 1, 2, 3, \tag {71}
$$
$$
P _ {j} \geq \phi_ {j} \left(P _ {X _ {1} \dots X _ {j}}, P _ {\hat {X} _ {1} \dots \hat {X} _ {j}}\right), \quad j = 1, 2, 3, \tag {72}
$$
for auxiliary random variables  $(X_{r,1}, X_{r,2}, X_{r,3})$  and  $(\hat{X}_1, \hat{X}_2, \hat{X}_3)$  such that
$$
\hat {X} _ {1} = \eta_ {1} \left(X _ {r, 1}\right), \quad \hat {X} _ {2} = \eta_ {2} \left(X _ {r, 1}, X _ {r, 2}\right), \quad \hat {X} _ {3} = X _ {r, 3}, \tag {73}
$$
$$
X _ {r, 1} \rightarrow X _ {1} \rightarrow \left(X _ {2}, X _ {3}\right), \tag {74}
$$
$$
X _ {r, 2} \rightarrow \left(X _ {2}, X _ {r, 1}\right)\rightarrow \left(X _ {1}, X _ {3}\right), \tag {75}
$$
$$
X _ {r, 3} \rightarrow \left(X _ {3}, X _ {r, 1}, X _ {r, 2}\right)\rightarrow \left(X _ {1}, X _ {2}\right), \tag {76}
$$
for some deterministic functions  $\eta_{1}(.)$  and  $\eta_{2}(.,)$
Theorem 4 For first-order Markov sources, a given  $(\mathsf{D},\mathsf{P})$  and  $\mathsf{R}\in \mathcal{R}(\mathsf{D},\mathsf{P})$ , we have
$$
\mathrm {R} + \log (\mathrm {R} + 1) + 5 \in \mathcal {R} ^ {o} (\mathrm {D}, \mathrm {P}). \tag {77}
$$
Moreover, the following holds:
$$
\mathcal {R} ^ {o} (\mathrm {D}, \mathrm {P}) \subseteq \mathcal {R} (\mathrm {D}, \mathrm {P}). \tag {78}
$$
Proof: Before stating the achievable scheme, we first discuss the strong functional representation lemma [35]. It states that for jointly distributed random variables  $X$  and  $Y$ , there exists a random variable  $U$  independent of  $X$ , and function  $\phi$  such that  $Y = \phi(X, U)$ . Here,  $U$  is not necessarily unique. The strong functional representation lemma states further that there exists a  $U$  which has information of  $Y$  in the sense that
$$
H (Y | U) \leq I (X; Y) + \log (I (X; Y) + 1) + 4. \tag {79}
$$
Notice that the strong functional representation lemma can be applied conditionally. Given  $P_{XY|W}$ , we can represent  $Y$  as a function of  $(X, W, U)$  such that  $U$  is independent of  $(X, W)$  and
$$
H (Y | W, U) \leq I (X; Y | W) + \log (I (X; Y | W) + 1) + 4. \tag {80}
$$
Proof of (77) (Inner bound):
For a given  $(\mathsf{D},\mathsf{P})$  and  $\mathsf{R}\in \mathcal{R}(\mathsf{D},\mathsf{P})$ , let  $\mathsf{X}_r = (X_{r,1},X_{r,2},X_{r,3})$  be jointly distributed with  $\mathsf{X} = (X_1,X_2,X_3)$  where the Markov chains (74)-(76) hold and the rate constraints in (68)-(70)
Figure 6: Strong functional representation lemma for  $T = 2$  frames.
are satisfied such that there exist  $(\hat{X}_1, \hat{X}_2, \hat{X}_3)$  for which distortion-perception constraints (71)-(72) hold. Denote the joint distribution of  $(\mathsf{X}, \mathsf{X}_r, \hat{\mathsf{X}})$  by  $P_{\mathsf{XX}_r\hat{\mathsf{X}}}$  and notice that according to the Markov chains in (74)-(76), it factorizes as the following
$$
\begin{array}{l} P _ {\mathbb {X X} _ {r} \hat {\mathsf {X}}} = P _ {X _ {1} X _ {2} X _ {3}} \cdot P _ {X _ {r, 1} | X _ {1}} \cdot P _ {X _ {r, 2} | X _ {r, 1} X _ {2}} \cdot P _ {X _ {r, 3} | X _ {r, 2} X _ {r, 1} X _ {3}} \\ \cdot \mathbb {1} \left\{\hat {X} _ {1} = g _ {1} \left(X _ {r, 1}\right) \right\} \cdot \mathbb {1} \left\{\hat {X} _ {2} = g _ {2} \left(X _ {r, 1}, X _ {r, 3}\right) \right\} \cdot \mathbb {1} \left\{\hat {X} _ {3} = X _ {r, 3} \right\}. \tag {81} \\ \end{array}
$$
For an illustration of encoded representations  $\mathsf{X}_r$  and reconstructions  $\hat{\mathsf{X}}$  in  $\mathcal{R}(\mathsf{D},\mathsf{P})$  which are induced by distribution  $P_{\mathbb{XX}_r\hat{\mathbf{X}}}$ , see Fig. 5.
Now, we show that  $\mathsf{R} + \log (\mathsf{R} + 1) + 5 \in \mathcal{R}(\mathsf{D},\mathsf{P})$ . The achievable scheme is as follows. Fix the joint distribution  $P_{\mathsf{X}_r}$  according to (81) which constructs the codebook, given by
$$
P _ {X _ {r}} = P _ {X _ {r, 1}} P _ {X _ {r, 2} \mid X _ {r, 1}} P _ {X _ {r, 3} \mid X _ {r, 2} X _ {r, 1}}. \tag {82}
$$
From the strong functional representation lemma [35], we know that
- there exist a random variable  $V_{1}$  independent of  $X_{1}$  and a deterministic function  $q_{1}$  such that  $X_{r,1} = q_{1}(X_{1},V_{1})$  and
$$
H \left(X _ {r, 1} \mid V _ {1}\right) \leq I \left(X _ {1}; X _ {r, 1}\right) + \log \left(I \left(X _ {1}; X _ {r, 1}\right) + 1\right) + 4, \tag {83}
$$
which means that the first encoder observes the source  $X_{1}$  and applies the function  $q_{1}$  to get  $X_{r,1}$  whose distribution needs to be preserved according to (82) (see Fig. 6);
- according to the conditional strong functional representation lemma, there exist a random variable  $V_{2}$  independent of  $(X_{2}, X_{r,1})$  and a deterministic function  $q_{2}$  such that  $X_{r,2} = q_{2}(X_{r,1}, X_{2}, V_{2})$  and
$$
H \left(X _ {r, 2} \mid X _ {r, 1}, V _ {2}\right) \leq I \left(X _ {2}; X _ {r, 2} \mid X _ {r, 1}\right) + \log \left(I \left(X _ {2}; X _ {r, 2} \mid X _ {r, 1}\right) + 1\right) + 4. \tag {84}
$$
At the second step, the representation  $X_{r,1}$  is available at the second encoder. So, upon observing the source  $X_{2}$ , it applies the function  $q_{2}$  to get  $X_{r,2}$  whose conditional distribution given  $X_{r,1}$  needs to be preserved according to (82) (see Fig. 6);
- according to the conditional strong functional representation lemma, there exist a random variable  $V_{3}$  independent of  $(X_{3}, X_{r,1}, X_{r,2})$  and a deterministic function  $q_{3}$  such that  $X_{r,3} = q_{3}(X_{r,1}, X_{r,2}, X_{3}, V_{3})$  and
$$
H \left(X _ {r, 3} \mid X _ {r, 1}, X _ {r, 2}, V _ {3}\right) \leq I \left(X _ {3}; X _ {r, 3} \mid X _ {r, 1}, X _ {r, 2}\right) + \log \left(I \left(X _ {3}; X _ {r, 3} \mid X _ {r, 1}, X _ {r, 2}\right) + 1\right) + 4. \tag {85}
$$
Now, the encoding and decoding are as follows
- With  $V_{1}$  available at all encoders and decoders, we can have a class of prefix-free binary codes indexed by  $V_{1}$  with the expected codeword length not larger than  $I(X_{1};X_{r,1}) + \log (I(X_{1};X_{r,1}) + 1) + 5$  to represent  $X_{r,1}$ , losslessly (see Fig. 6).
- With  $V_{2}$  available at the encoders and decoders, we can design a set of prefix-free binary codes indexed by  $(V_{2}, X_{r,1})$  with expected codeword length not larger than  $I(X_{2}; X_{r,2}|X_{r,1}) + \log(I(X_{2}; X_{r,2}|X_{r,1}) + 1) + 5$  to represent  $X_{r,2}$ , losslessly (see Fig. 6).
- Similarly, one can represent  $X_{r,3}$  losslessly with  $V_{3}$  available at the third encoder and decoder.
- The decoders can use functions  $\hat{X}_1 = \eta_1(X_{r,1})$ ,  $\hat{X}_2 = \eta_2(X_{r,1}, X_{r,2})$  and  $\hat{X}_3 = X_{r,3}$  to get the reconstruction  $\hat{X}$ .
This shows that  $\mathsf{R} + \log (\mathsf{R} + 1) + 5 \in \mathcal{R}^o (\mathsf{D},\mathsf{P})$ .
Proof of (78) (Outer Bound):
For any  $(\mathsf{D},\mathsf{P}),\mathsf{R}\in \mathcal{R}^o (\mathsf{D},\mathsf{P})$ , shared randomness  $K$ , encoding functions  $f_{j}\colon \mathcal{X}_{1}\times \ldots \times \mathcal{X}_{j}\times \mathcal{K}\to$ $\mathcal{M}_j$  and decoding functions  $g_{j}\colon \mathcal{M}_{1}\times \mathcal{M}_{2}\times \ldots \times \mathcal{M}_{j}\times \mathcal{K}\to \hat{\mathcal{X}}_{j}$  such that
$$
R _ {j} \geq \mathbb {E} [ \ell (M _ {j}) ], \quad j = 1, 2, 3, \tag {86}
$$
$$
D _ {j} \geq \mathbb {E} \left[ \| X _ {j} - \hat {X} _ {j} \| ^ {2} \right], \quad j = 1, 2, 3, \tag {87}
$$
$$
P _ {j} \geq \phi_ {j} \left(P _ {X _ {1} \dots X _ {j}}, P _ {\hat {X} _ {1} \dots \hat {X} _ {j}}\right), \quad j = 1, 2, 3, \tag {88}
$$
we lower bound the expected length of the messages. Define
$$
X _ {r, 1} := \left(M _ {1}, K\right), \tag {89}
$$
$$
X _ {r, 2} := \left(M _ {1}, M _ {2}, K\right), \tag {90}
$$
and recall that according to the decoding functions, we have
$$
\hat {X} _ {j} = g _ {j} \left(M _ {1}, \dots , M _ {j}, K\right), \quad j = 1, 2, 3. \tag {91}
$$
We can write
$$
\begin{array}{l} R _ {1} \geq \mathbb {E} [ \ell (M _ {1}) ] \geq H (M _ {1} | K) (92) \\ = I \left(X _ {1}; M _ {1} \mid K\right) (93) \\ = I \left(X _ {1}; M _ {1}, K\right) (94) \\ = I \left(X _ {1}; X _ {r, 1}\right). (95) \\ \end{array}
$$
Now, consider the following set of inequalities
$$
\begin{array}{l} R _ {2} \geq \mathbb {E} [ \ell (M _ {2}) ] \geq H (M _ {2} | M _ {1}, K) (96) \\ = I \left(X _ {1}, X _ {2}; M _ {2} \mid M _ {1}, K\right) (97) \\ = I \left(X _ {1}, X _ {2}; X _ {2, r} \mid X _ {r, 1}\right). (98) \\ \end{array}
$$
$$
\begin{array}{l} R _ {3} \geq \mathbb {E} [ \ell (M _ {3}) ] \geq H \left(M _ {3} \mid M _ {1}, M _ {2}, K\right) (99) \\ = I \left(X _ {1}, X _ {2}, X _ {3}; M _ {3} \mid M _ {1}, M _ {2}, K\right) (100) \\ \geq I \left(X _ {1}, X _ {2}, X _ {3}; \hat {X} _ {3} \mid X _ {r, 1}, X _ {r, 2}\right). (101) \\ \end{array}
$$
Notice that the definitions in (89)-(90) imply the following Markov chains
$$
X _ {r, 1} \rightarrow X _ {1} \rightarrow \left(X _ {2}, X _ {3}\right), \tag {102}
$$
$$
X _ {r, 2} \rightarrow \left(X _ {1}, X _ {2}, X _ {r, 1}\right)\rightarrow X _ {3}. \tag {103}
$$
On the other hand, the decoding functions of the first and second steps imply that
$$
\hat {X} _ {1} = g _ {1} \left(M _ {1}, K\right), \tag {104}
$$
$$
\hat {X} _ {2} = g _ {2} \left(M _ {1}, M _ {2}, K\right), \tag {105}
$$
where together with definitions in (89) and (90), we can write
$$
\hat {X} _ {1} = g _ {1} \left(M _ {1}, K\right) := \eta_ {1} \left(X _ {r, 1}\right), \tag {106}
$$
$$
\hat {X} _ {2} = g _ {2} \left(M _ {1}, M _ {2}, K\right) := \eta_ {2} \left(X _ {r, 1}, X _ {r, 2}\right), \tag {107}
$$
such that  $\eta_1(.)$  and  $\eta_2(.,\cdot)$  are deterministic functions.
Now, consider the fact that the set of constraints in (87)-(88), (95), (98), (101) with Markov chains in (102)-(103) and deterministic functions in (106)-(107) constitute an iRDP region, denoted by  $\bar{C}_{\mathrm{RDP}}$  which is the set of all tuples  $(\mathsf{R},\mathsf{D},\mathsf{P})$  such that
$$
R _ {1} \geq I \left(X _ {1}; X _ {r, 1}\right), \tag {108}
$$
$$
R _ {2} \geq I \left(X _ {1}, X _ {2}; X _ {r, 2} \mid X _ {r, 1}\right), \tag {109}
$$
$$
R _ {3} \geq I \left(X _ {1}, X _ {2}, X _ {3}; \tilde {X} _ {3} \mid X _ {r, 1}, X _ {r, 2}\right), \tag {110}
$$
$$
D _ {j} \geq \mathbb {E} [ \| X _ {j} - \hat {X} _ {j} \| ^ {2} ], \quad j = 1, 2, 3, \tag {111}
$$
$$
P _ {j} \geq \phi_ {j} \left(P _ {X _ {1} \dots X _ {j}}, P _ {\hat {X} _ {1} \dots \hat {X} _ {j}}\right), \quad j = 1, 2, 3, \tag {112}
$$
for auxiliary random variables  $(X_{r,1},X_{r,2})$  and  $(\hat{X}_1,\hat{X}_2,\hat{X}_3)$  satisfying the following
$$
\hat {X} _ {1} = \eta_ {1} \left(X _ {r, 1}\right), \quad \hat {X} _ {2} = \eta_ {2} \left(X _ {r, 1}, X _ {r, 2}\right) \tag {113}
$$
$$
X _ {r, 1} \rightarrow X _ {1} \rightarrow \left(X _ {2}, X _ {3}\right), \tag {114}
$$