Linear Interactive Encoding and Decoding Schemes for Lossless Source Coding with Decoder Only Side Information

Near lossless source coding with side information only at the decoder, was first considered by Slepian and Wolf in 1970s, and rediscovered recently due to applications such as sensor network and distributed video coding. Suppose X is a source and Y is the side information. The coding scheme proposed by Slepian and Wolf , called SW coding, in which information only flows from the encoder to the decoder, was shown to achieve the rate H(X|Y ) asymptotically for stationary ergodic source pairs. As H(X|Y ) is the minimum achievable rate even if the encoder is informed of the side information Y , this implies the optimality of SW coding for any stationary ergodic source-side information pairs. However, it is shown by Yang and He that SW coding can not achieve this rate for most of non-ergodic source-side information pairs. Recently, a new source coding paradigm called interactive encoding and decoding(IED) was proposed for near lossless coding with side information only at the decoder. In such paradigm, information flows in both ways, from the encoder to the decoder and vice verse, and the behaviour of the encoder also relies on the bits sent back from the decoder. The compression rate of an IED scheme is defined as average number of bits per symbol exchanged between the encoder and the decoder. In contrary to SW coding, IED can achieve the rate of H(X|Y ) for any stationary source-side information pairs, ergodic or not. Also it was shown that for memoryless source-side information pairs, IED schemes achieve better redundancy performance than SW coding schemes. And universal IED can be built coupled with classical universal codes for any stationary ergodic source-side information pairs, while it is well known that there does not exist a universal SW coding scheme. The results reviewed above show that IED schemes are much more appealing than SW coding schemes to applications where the interaction between the encoder and the decoder is possible. However, the IED schemes proposed by Yang and He do not have an intrinsic structure that is amenable to design and implement in practice. Towards practical design, we restrict the encoding method to linear block codes, resulting in linear IED schemes. It is then shown that this restriction will not undermine the asymptotical performance of IED, in the sense that a sequence of linear IED schemes can be always found for any stationary ergodic source-side information pair (X, Y ) to achieve the rate of conditional entropy H(X|Y ). Moreover, the result can also be extended to non-ergodic source-side information pairs. Another step of practical design of IED schemes is to make the computational complexity incurred by encoding and decoding feasible. In the framework of linear IED, a scheme can be conveniently described by parity check matrices. It can be easily observed that density of these matrices (measured as the average number of non-zero entries) is directly related to the scheme’s encoding and decoding complexity: the complexity increases as the density increases. Further, we get an interesting trade-off between the density of the associated parity check matrices and the resulting symbol error probability. Our analysis reveals that as long as np ∗ n = Ω(log n/n), where n and p ∗ n are real numbers, one can always construct a