The number of choices is at a minimum if at each stage we arrange to choose between two equally likely possibilities. The theory of information is first and foremost a linguistic tool. It enables us to speak precisely and quantitatively where previously we have had to speak vaguely and qualitatively. The general information theory is concerned with the problem of measuring changes in knowledge. It key is the fact that we can represent what we know by means of pictures, logical statements, symbolic models, etc.
Communication is the activity of replicating information.
Communication engineers have not developed a concept of information at all. Semanticists on the other hand are concerned with different features of information, such as its logical complexity and its meaning. But both engineers and semanticists have in mind the same concept of information. McKay argues that the connection between statistical and semantic features of information cannot but be indirect, these are features of one and the same central concept. Fundamentally, it implies that in some circumstance or other my expectations will be different.