Redundancy-Optimal FF Codes for a General Source and Its Relationships to the Rate-Optimal FF Codes
In this paper we consider fixed-to-fixed length (FF) coding of a general source X with vanishing error probability and define two kinds of optimalities with respect to the coding rate and the redundancy, where the redundancy is defined as the difference between the coding rate and the symbolwise ideal codeword length. We first show that the infimum achievable redundancy coincides with the asymptotic width W(X) of the entropy spectrum. Next, we consider the two sets C-(H) over bar(X) and C-W(X) and investigate relationships between them, where C-(H) over bar(X) and C-W(X) denote the sets of all the optimal FF codes with respect to the coding rate and the redundancy, respectively. We give two necessary and sufficient conditions corresponding to C-(H) over bar(X) subset of C-W(X) and C-W(X) subset of C-(H) over bar(X), respectively. We can also show the existence of an FF code that is optimal with respect to both the redundancy and the coding rate.
redundancy general sources coding rate fixed-to-fixed length source coding information-spectrum methods