After reviewing the strengthened hypotheses of the Shannon-MacMillan-Breiman Theorem, versus the standard statement of the Noiseless Coding Theorem, we state and prove a similar result for relative information , otherwise known as information gain. If the gain results from receiving a side message, this gain, when averaged over the ensemble of possible side messages, is precisely the pragmatic information defined in Weinberger ( 2002 ). The relative information result proven herein can be used to extend the information theoretic analysis of the Kelly Criterion, and its generalization, the horse race, an analysis of securities market trading strategies presented in Cover and Thomas (1992). We show, in particular, that their results for statistically independent horse races also apply to a series of races where the stochastic process of winning horses, payoffs, and strategies depend on some ergodic process, including, but not limited to the history of previous races. Also, if the bettor is receiving messages (side information) about the probability distribution of winners, the doubling rate of the bettor's winnings can be interpreted as the pragmatic information of the messages. Both the theorem proven herein and the application to trading make a compelling case for Weinberger's definition of pragmatic information .
This paper is part of an ongoing investigation of "pragmatic information", defined in Weinberger ( 2002 ) as "the amount of information actually used in making a decision". Because a study of information rates led to the Noiseless and Noisy Coding Theorems, two of the most important results of Shannon's theory, we begin the paper by defining a pragmatic information rate, showing that all of the relevant limits make sense, and interpreting them as the improvement in compression obtained from using the correct distribution of transmitted symbols. The first of two applications of the theory extends the information theoretic analysis of the Kelly Criterion, and its generalization, the horse race, to a series of races where the stochastic process of winning horses, payoffs, and strategies depend on some stationary process, including, but not limited to the history of previous races. If the bettor is receiving messages (side information) about the probability distribution of winners, the doubling rate of the bettor's winnings is bounded by the pragmatic information of the messages. A second application is to the question of market efficiency. An efficient market is, by definition, a market in which the pragmatic information of the "tradable past" with respect to current prices is zero. Under this definition, markets whose returns are characterized by a GARCH(1,1) process cannot be efficient. Finally, a pragmatic informational analogue to Shannon's Noisy Coding Theorem suggests that a cause of market inefficiency is that the underlying fundamentals are changing so fast that the price discovery mechanism simply cannot keep up. This may happen most readily in the run-up to a financial bubble, where investors' willful ignorance degrade the information processing capabilities of the market .
R
After reviewing the strengthened hypotheses of the Shannon-MacMillan-Breiman Theorem, versus the standard statement of the Noiseless Coding Theorem, we state and prove a similar result for relative information , otherwise known as information gain. If the gain results from receiving a side message, this gain, when averaged over the ensemble of possible side messages, is precisely the pragmatic information defined in Weinberger (
This paper is part of an ongoing investigation of "pragmatic information", defined in Weinberger (
R
). The relative information result proven herein can be used to extend the
) as "the amount of information actually used in making a decision". Because a study of information rates led to the Noiseless and Noisy Coding Theorems, two of the most important results of Shannon's theory, we begin the paper by defining a pragmatic information rate, showing that all of the relevant limits make sense, and interpreting them as the improvement in compression obtained from using the correct distribution of transmitted symbols. The first of two applications of the theory extends the
D
an analysis of securities market trading strategies presented in Cover and Thomas (1992). We show, in particular, that their results for statistically independent horse races also apply
R
ergodic
stationary
R
Also, if
If
R
can be interpreted as
is bounded by
R
Both the theorem proven herein and the application to trading make a compelling case for Weinberger's definition of pragmatic information
A second application is to the question of market efficiency. An efficient market is, by definition, a market in which the pragmatic information of the "tradable past" with respect to current prices is zero. Under this definition, markets whose returns are characterized by a GARCH(1,1) process cannot be efficient. Finally, a pragmatic informational analogue to Shannon's Noisy Coding Theorem suggests that a cause of market inefficiency is that the underlying fundamentals are changing so fast that the price discovery mechanism simply cannot keep up. This may happen most readily in the run-up to a financial bubble, where investors' willful ignorance degrade the information processing capabilities of the market