A typical problem is to judge how relevant a document is to a given query or agent profile. Bayesian theory aids in this calculation by relating this judgement to details that we already know,such as the model of an agent. Concepts important to sets of documents can be determined, allowing new documents to be accurately classified. Shannon's Information Theory Information Theory is the mathematical foundation for all digital communications systems. This result is then extended, by more sophisticated mathematical arguments, to when units are related.
Natural languages contain a high degree of redundancy.
Information theory provides a framework for extracting the concepts from the redundancy. Autonomy's approach to concept modelling relies on Shannon's theory that the less frequently a unit of communication occurs, the more information it conveys. Therefore ideas which are more rare within the context of a communication, tend to be more indicative of its meaning.