On the one hand, if individuals possess an explicit utility function stemming from the rate of production of expected offspring, they can readily adapt to novelty in a two-armed bandit problem. Embedding such a function in a simple rule of thumb involving no beliefs about prior or posterior probabilities leads to maximization of expected offspring, in a certain limit as the number of repetitions tends to infinity. In general, on the other hand, if any rule yields such evolutionary optimality of behavior, this biological utility function is implicit at least. The Evolution of Intelligence and The Red Queen. This paper models the evolution of strategic complexity as an arms race.
Strategies are noisy and have bounded recall.
A suitable equilibrium can be shown to exist. Relative to any such equilibrium, sufficiently greater rationality enjoys an edge which is not decreasing in the original level of recall, thus capturing the Red Queen effect. The selection pressure to reduce a small amount of noise must be less than that to increase recall.