Abstract-A quantity called the finite-state complexity is assigned to every infinite sequence of elements drawn from a finite set. This quantity characterizes the largest compression ratio that can be achieved in accurate transmission of the sequence by any finite-state encoder (and decoder). Coding theorems and converses are derived for an individual sequence without any probabilistic characterization, and universal data compression algorithms are introduced that are asymptotically optimal for all sequences over a given alphabet. The finite-state complexity of a sequence plays a role similar to that of entropy in classical information theory (which deals with probabilistic ensembles of sequences rather than an individual sequence). For a probabilistic source, the expectation of the finite state complexity of its sequences is equal to the source's entropy. The finite state complexity is of particular interest when the source statistics are unspecified.
First, the procedures achieve a compression length that is asymptotically equal to the so-called finite-state complexity, which is the minimum length that can be achieved by finite-state encoding/decoding procedures REF .
Coding theorems for individual sequences
IEEE Trans. Inf. Theory
IEEE Trans. Inf. Theory
Computer Science Mathematics