# Beliefs at different timescales

Why is a chess game the opposite of an ideal gas? On short timescales an ideal gas is described by elastic collisions. And a single move in chess can be modeled by a [policy network](https://arxiv.org/abs/1712.01815).

The difference is in long timescales: If we simulated elastic collisions for a long time, we'd end up with a [complicated distribution](https://www.lesswrong.com/posts/QkX2bAkwG2EpGvNug/the-second-law-of-thermodynamics-and-engines-of-cognition) over the microstates of the gas. But we can't run simulations for a long time, so we have to make do with the [Boltzmann distribution](https://en.wikipedia.org/wiki/Boltzmann_distribution), which is a lot less accurate.

Similarly, if we rolled out our policy network to get a distribution over chess game outcomes (win/loss/draw), we'd get the distribution of outcomes under self-play. But if we're observing a game between two players who are better players than us, we have access to a [more accurate model](https://www.lesswrong.com/posts/rEDpaTTEzhPLz4fHh/expected-creative-surprises) based on their [Elo ratings](https://en.wikipedia.org/wiki/Elo_rating_system).

Can we formalize this? Suppose we're observing a chess game. Our beliefs about the next move are conditional probabilities of the form $P_1(x_{k+1} | x_0 \cdots x_k)$, and our beliefs about the next $n$ moves are conditional probabilities of the form $P_n(x_{k+1} \cdots x_{k+n} | x_0 \cdots x_k)$. We can transform beliefs of one type into the other using the operators

$$ (\Pi_n P_1)(x_{k+1} \cdots x_{k+n} | x_0 \cdots x_k) := \prod_{i=0}^{n-1} P_1(x_{k+i+1} | x_0 \cdots x_{k+i}) $$

$$ (\Sigma^n P_n)(x_{k+1} | x_0 \cdots x_k) := \sum_{x_{k+2}} \cdots \sum_{x_{k+n}} P_n(x_{k+1} \cdots x_{k+n} | x_0 \cdots x_k) $$

If we're logically omniscient, we'll have $\Pi_n P_1 = P_n$ and $\Sigma^n P_n = P_1$. But in general we will not. A chess game is short enough that $\Pi_n$ is easy to compute, but $\Sigma^n$ is too hard because it has exponentially many terms. So we can have a long-term model $P_n$ that is more accurate than the rollout $\Pi_n P_1$, and a short-term model $P_1$ that is less accurate than $\Sigma^n P_n$. This is a sign that we're dealing with an intelligence: We can predict outcomes better than actions.

If instead of a chess game we're predicting an ideal gas, the relevant timescales are so long that we can't compute $\Pi_n$ or $\Sigma^n$. Our long-term thermodynamic model $P_n$ is less accurate than a simulation $\Pi_n P_1$. This is often a feature of reductionism: Complicated things can be reduced to simple things that can be modeled more accurately, although more slowly.

In general, we can have several models at different timescales, and $\Pi$ and $\Sigma$ operators connecting all the levels. For example, we might have a short-term model describing the physics of fundmental particles; a medium-term model describing a person's motor actions; and a long-term model describing what that person accomplishes over the course of a year. The medium-term model will be less accurate than a rollout of the short-term model, and the long-term model may be more accurate than a rollout of the medium-term model if the person is smarter than us.