Information, Constraint, and Meaning

· Despite the familiar and correct discl3imer that information theory (Shannon and Weaver, 1949) does not concern the semantic level of communication, the technical definition of information nonetheless bears directly and importantly on the subject of meaning. Meaning, at least in one sense of the word, is the recognition of constraint and is based on isomorphism of structure. Constraint reduces information yet information is also the very ~ub1 strate of meaning. Meaning is thus the union of the informative and the intelligible (Moles, 1958), the reconciliation of this dialectical opposition being achievable in several different ways. It is commonly understood that "information" in the mathematical theory of communication (Shannon and Weaver, 1949) refers only to the technical as? ects of signal transmission and not at a~l to questions of meaning. To deal with such questions, Bar-Hillel and Carnap (1953, 1964) developed a semantic theory of information formally similar to ~hannon's theory but based upon a l? gical, as opposed to statistical, notion of probability. The Shannon entropy expression, H = -! pJ. log p., . h J in t e semantic theory is used for the average uncertainty associated with a set of possible meanings, as compared in the technical theory, to the average' uncertainty associated with a set of (emit~ed or received) symbols. Both the technical and semantic theories are thus essentially theories of selection, But by virtue of this formal similarity, the technical and semantic levels re~ain fully distinct and unbridgeable. This complete separability of levels pre~ludes a~y quantitative linkage between information and meaning, though meaning depends upon the receipt of information a~d if meaning is to be quantified, one' might ex9ect that its amount should be a f~nction ~f (and possibly vary monotonically with) the amount of information. The relationship, however, must be more complex since from a random sequence of symbols one gains maximum information, but no meaning. Indeed, Weaver remarked on his vague feeling that information and mean~ng may prove to be something like a pair of canonically conjugate variables in quantum theory . subject to some joint restriction that condemns a person to the sacrifice of the one as he insists on having much of the other • . (Shannon & Weaver, 1949, p 28) but gave no specific basis for such a complementarity. The purpose of this note is to pursue this suggestion further and by doing so demonstrate a complex relationship between infor~ation, constraint, and meaning. Information, for Shannon and Weavec, is reduction in uncertainty i c -~u = uinitial ufinal' If a definite messase is received, i.e., the final uncertain~·; is zero, then the information gained is. equal to the initial uncertainty. Messages from a source about which there is complete initial uncertaintv, i.e., a random source, carry maximum infoc~a:ion. Thus, for example, if a sequence of symbols s 1 , s 2 , . sn is recei•1ed, one gains maximal information when receio" of sj in no way reduces the uncertai;ty of what sj+l (and succeeding symbols) will be. To the extent that the source is nonrandom and there exist cor.strain"s between symbols at different times, the information provided by the message is reduced (Shannon and Weaver, 1949; ~iller,