# Supplemental Materials

Proof of The Maximization of The Mutual Information, \( \operatorname { I } [ x ^ { t } ; y ^ { t } ] : \) In this section, we prove that the maximization of the mutual information,\( \operatorname { I } [ x ^ { t } ; y ^ { t } ] \), and Eq.(8) are equivalent to equality in Eq.(3), under suitable conditions. First, we note that we can replace \( \ensuremath { \dot { \nu } } ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \) by \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } , y ^ { t + 1 } ) \) in Eqs.(3), (4), (5), (6) and (7). In this case, equality in Eq.(3) follows from equality in the Jensen inequality \( ( \exp F ( Z ) = \operatorname { E } [ \exp F ( Z ) ] \) with probability 1):

\[ e ^ { - I _ { t r } ^ { t + 1 } - \widehat { \Theta } _ { \nu } ^ { t } } = \mathrm { E } [ e ^ { - I _ { t r } ^ { t + 1 } - \widehat { \Theta } _ { \nu } ^ { t } } ] = 1 . \eqno ( 1 4 ) \]

This implies −It+1 bttr −Θν = 0 with probability 1. By rearranging terms, we have

\[ \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } , y ^ { t + 1 } ) = p _ { s } ( x ^ { t } | y ^ { t + 1 } , y ^ { t } , x ^ { t + 1 } ) . \eqno ( 1 5 ) \]

Hence, by including \( y ^ { t + 1 } \) as a conditioning variable of \( \nu \), we can easily obtain the equality. However, by reducing the number of conditioning variables, we can also obtain the maximization of the mutual information,\( \operatorname { I } [ x ^ { t } ; y ^ { t } ] \), as we noted in the main text. We prove this in the following.

First, we obtain an explicit expression of the optimal \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \) in Eq.(8) from the following inequality:

\[ \mathbb { E } \left[ \log \frac { \nu ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } \right] = \sum _ { u ^ { t } , x ^ { t + 1 } } p _ { s } ( y ^ { t } , x ^ { t + 1 } ) \sum _ { x ^ { t } } p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) \log \frac { \nu ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } \leq 0 . \qquad \qquad \qquad ( 1 6 ) \]

The above inequality is derived from the inequality \( F - 1 \geq \log F \) for positive real F, and thus, the optimality condition in Eq.(8) is obtained from the equality, \( F - 1 = \log F \leftrightarrow F = 1 \) with probability 1:

\[ \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) = p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) . \eqno ( 1 7 ) \]

Then, in order to analyze the equality condition of Eq.(3) for \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \), we calculate the difference \( \Delta \) between the values of −E[Θtν] as calculated with Eqs. (17) and (15), writing

\[ \Delta = \operatorname { E } \left[ \log { \frac { p _ { s } ( x ^ { t } | y ^ { t } , y ^ { t + 1 } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } } \right] = \operatorname { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] . \eqno ( 1 8 ) \]

Because −E[Θtν] = Ix→y in the case with Eq.(15), we obtain

\[ \begin{array} { r } { \mathrm { I } _ { x \to y } + \mathrm { E } [ \Theta _ { \nu } ^ { t } ] = \mathrm { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] , \eqno ( 1 9 ) } \end{array} \]

in the case with Eq.(17).

Next, we show that the condition

\[ \begin{array} { r } { \mathrm { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] = 0 , \eqno ( 2 0 ) } \end{array} \]

is equivalent to the maximization of the mutual information,

\[ \begin{array} { r } { \mathrm { I } [ x ^ { t } ; y ^ { t } ] = \mathrm { H } [ y ^ { t } ] , \eqno ( 2 1 ) } \end{array} \]

on the assumption that the environmental state space Y is not too finely partitioned in comparison with the precision of neural control over the environment. Precisely, we assume that there is no coarse-grained partition \( y ^ { \prime } \) of Y such that the neural control has the same precision on the two partitions Y and \( \mathcal { V } ^ { \prime } \) of the environmental state space. We also assume that \( p _ { s } ( y ^ { t + 1 } | y ^ { t } ) \neq 1 \) for all \( y ^ { t } \) and \( y ^ { t + 1 } \), which holds for most sets of values of the model parameters in a general model. A coarse-grained partition \( y ^ { \prime } \) is a set of subcollections of Y such that \( y ^ { \prime } \cap y ^ { \prime \prime } = \emptyset \) for any \( y ^ { \prime } \neq y ^ { \prime \prime } \in \mathcal { y } ^ { \prime } \) and \( \cup _ { y ^ { \prime } \in \mathcal { y ^ { \prime } } } y ^ { \prime } = \mathcal { y } \). For any such coarse-grained partition \( \mathcal { V } ^ { \prime } \), we require

\[ \begin{array} { r } { \mathrm { I } [ y ^ { t + 1 , \prime } ; x ^ { t } | y ^ { t } ] \neq \mathrm { I } [ y ^ { t + 1 } ; x ^ { t } | y ^ { t } ] , \; y ^ { t + 1 } \in y ^ { t + 1 , \prime } \in \mathcal { Y } ^ { \prime } , \eqno ( 2 2 ) } \end{array} \]

where we have defined the random variable \( y ^ { t + 1 , \prime } \), which takes values in \( \mathcal { V } ^ { \prime } \) with \( y ^ { t + 1 } \in y ^ { t + 1 , \prime } . \). Under this assump-tion, we first show that the conditional mutual information,

\[ \mathrm { I } [ y ^ { t + 1 } ; x ^ { t + 1 } | y ^ { t } ] = \sum _ { x ^ { t + 1 } , y ^ { t + 1 } , y ^ { t } } p _ { s } ( y ^ { t } , y ^ { t + 1 } , x ^ { t + 1 } ) \log \frac { p _ { s } ( y ^ { t + 1 } | x ^ { t + 1 } , y ^ { t } ) } { p _ { s } ( y ^ { t + 1 } | y ^ { t } ) } , \eqno ( 2 3 ) \]

must be maximal. Note that the conditional mutual information takes its maximal value and hence satisfies

\[ \begin{array} { r } { \mathrm { I } [ x ^ { t + 1 } ; y ^ { t + 1 } | y ^ { t } ] = \mathrm { H } [ y ^ { t + 1 } | y ^ { t } ] \eqno ( 2 4 ) } \end{array} \]

if and only if \( p _ { s } ( y ^ { t + 1 } | x ^ { t + 1 } , y ^ { t } ) \ = \ 1 \) with probability 1. Thus, to obtain the desired result, we show that \( p _ { s } ( y ^ { t + 1 } | x ^ { t + 1 } , y ^ { t } ) > 0 \) for multiple \( y ^ { t + 1 } \) with some \( y ^ { t } = y _ { 0 } \) and \( { \boldsymbol { x } } ^ { t + 1 } = { \boldsymbol { x } } _ { 0 } \) contradicts Eq.(20).

First, we define the set

\[ \overline { { y } } = \{ y ^ { t + 1 } \in \mathcal { Y } | p _ { s } ( y ^ { t + 1 } | x ^ { t + 1 } = x _ { 0 } , y ^ { t } = y _ { 0 } ) > 0 \} , \eqno ( 2 5 ) \]

and a coarse-grained partition of \( _ { 3 } \) as

\[ \mathcal { V } = \{ \{ y \} \} _ { y \in \mathcal { V } \backslash \overline { { y } } } \cup \{ \overline { { y } } \} . \eqno ( 2 6 ) \]

Here, \( y \setminus \overline { y } \) is the relative complement of \( \overline { { y } } \) in \( y \), which consists of all the elements of \( y \) that are not contained in y. The assumption in Eq.(22) requires

\[ \begin{array} { r l r } { \operatorname { I } [ y ^ { t + 1 } ; x ^ { t } | y ^ { t } ] - \operatorname { I } [ y ^ { t + 1 , \prime } ; x ^ { t } | y ^ { t } ] = \operatorname { I } [ y ^ { t + 1 } , y ^ { t + 1 , \prime } ; x ^ { t } | y ^ { t } ] - \operatorname { I } [ y ^ { t + 1 , \prime } ; x ^ { t } | y ^ { t } ] } & { } & \\ { = \operatorname { I } [ y ^ { t + 1 } ; x ^ { t } | y ^ { t + 1 , \prime } , y ^ { t } ] } & { } & { } \\ { > 0 . } & { } & { \qquad \qquad \qquad \qquad \qquad ( 2 7 \times ( 3 \times 3 ) + 1 ) } \end{array} \]

where the first equality holds because \( y ^ { t + 1 } \) uniquely determines \( y ^ { t + 1 , \prime } \) and thus the additional inclusion of \( y ^ { t + 1 , \prime } \) in the first term does not affect the value of the conditional mutual information. Also, Eq.(20) implies

\[ \begin{array} { r l r } { { \mathrm { I } } [ y ^ { t + 1 } ; x ^ { t } | y ^ { t } , x ^ { t + 1 } ] = { \mathrm { I } } [ y ^ { t + 1 } ; x ^ { t } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ] = 0 . \eqno ( 2 8 ) } \end{array} \]

Now, recall that the inclusion of additional conditioning variables (in this case, \( y ^ { t + 1 , \prime } ) \) always reduces the value of the conditional mutual information. The right-hand side of the above equation can be written as

\[ \begin{array} { r l } & { \quad \mathrm { E } \left[ \log \frac { p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( y ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } \right] } \\ & { = \mathrm { E } \left[ \log \left\{ \frac { p _ { s } ( x ^ { t + 1 } | y ^ { t + 1 } ) p _ { s } ( y ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t } ) } { \sum _ { \tilde { y } ^ { t + 1 } \in \mathcal { Y } } p _ { s } ( x ^ { t + 1 } | y ^ { t + 1 } ) p _ { s } ( \tilde { y } ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } ) } \right\} \right] } \\ & { = 0 , } \end{array} \]

with the dummy variables \( \widetilde { \boldsymbol { y } } ^ { t + 1 } \) and \( \widehat { y } ^ { t + 1 } \) having the same (conditional) distributions as \( y ^ { t + 1 } \). The above equality requires that the argument of the logarithm be 1 with probability 1, since \( F - 1 \geq \log F \), \( F - 1 = \log F \leftrightarrow F = 1 \) and

\[ \begin{array} { r l } & { \quad - \mathrm { E } \left[ \log \frac { p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( y ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } \right] } \\ & { = \underbrace { \sum _ { x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } , y ^ { t } , x ^ { t + 1 } \} \sum _ { y ^ { t + 1 } } p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) \log \frac { p _ { s } ( y ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } } \\ & { \leq \underbrace { \sum _ { x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } } p _ { s } ( x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) \sum _ { y ^ { t + 1 } } p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) \left( \frac { p _ { s } ( y ^ { t + 1 } | y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( y ^ { t + 1 } | x ^ { t } , y ^ { t + 1 , \prime } , y ^ { t } , x ^ { t + 1 } ) } - 1 \right) } _ { ( 3 0 ) } } \\ & { = 0 . } \end{array} \]