But information theory describes mathematically the sources, channels, and outputs used in communications. Its origins are in telegraphy and it is now a field closely related to computer science. One of the first tasks of information theory was to formulate rules for maximizing the speed of transmission of information.
Academic protocol calls for including more than one bit in the communication.
A compressed version of such a letter could be one bit long, however, essentially without loss of information. This talk will take up Shannon's results and others and related them to problems of data compression. We will look at the Huffman coding method, the Ziv Lempel algorithm, and some related areas of study, including definitions of randomness, complexity of objects, and algorithmic information theory. Here randomness is taken as an inherent feature of an object rather than a feature of the process by which an object is obtained. A relatively new branch of study related to our topic, one that is of special interest in computer science, is called algorithmic information theory.