Although the material in the two courses is related, we think it unwise for information theory to be a prerequisite for data compression, and have written the data compression section of the text so that it can be read by or presented to students with no prior knowledge of information theory. There are references in the data compression part to results and proofs in the information theory part of the text, and those who are interested may browse over those references, but it is not absolutely necessary to do so. Our main aim in the data compression part of the text, as well as in the course it grew from, is to acquaint the students with a number of significant lossless compression techniques, and to discuss two lossy compression methods. Our aim is for the students to emerge competent in and broadly conversant with a large range of techniques. Nonetheless, proofs are provided, sometimes in the text, sometimes in the exercises, so that the instructor may have the option of emphasizing the mathematics of data compression to some degree.
Information theory is of a more theoretical nature than data compression.
It provides a vocabulary and a certain abstraction that can bring the power of simplification to many different situations. We thought it reasonable to treat it as a mathematical theory, and to present the fundamental definitions and elementary results of that theory in utter abstraction from the particular problems of communication through noisy channels, which inspired the theory in the first place. We bring the theory to bear on noisy channels in Chapters 3 and 4. The treatment of information theory given here is extremely elementary. We feel that this elementary approach is appropriate for the target audience, and that, by leaving more complicated sources and channels out of the picture, we more effectively impart the grasp of Information Theory that we hope our students will take with them.