# Supplemental Materials

Proof of The Maximization of The Mutual Information, \( \operatorname { I } [ x ^ { t } ; y ^ { t } ] : \) In this section, we prove that the maximization of the mutual information,\( \operatorname { I } [ x ^ { t } ; y ^ { t } ] \), and Eq.(8) are equivalent to equality in Eq.(3), under suitable conditions. First, we note that we can replace \( \ensuremath { \dot { \nu } } ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \) by \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } , y ^ { t + 1 } ) \) in Eqs.(3), (4), (5), (6) and (7). In this case, equality in Eq.(3) follows from equality in the Jensen inequality \( ( \exp F ( Z ) = \operatorname { E } [ \exp F ( Z ) ] \) with probability 1):

\[ e ^ { - I _ { t r } ^ { t + 1 } - \widehat { \Theta } _ { \nu } ^ { t } } = \mathrm { E } [ e ^ { - I _ { t r } ^ { t + 1 } - \widehat { \Theta } _ { \nu } ^ { t } } ] = 1 . \eqno ( 1 4 ) \]

This implies −It+1 bttr −Θν = 0 with probability 1. By rearranging terms, we have

\[ \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } , y ^ { t + 1 } ) = p _ { s } ( x ^ { t } | y ^ { t + 1 } , y ^ { t } , x ^ { t + 1 } ) . \eqno ( 1 5 ) \]

Hence, by including \( y ^ { t + 1 } \) as a conditioning variable of \( \nu \), we can easily obtain the equality. However, by reducing the number of conditioning variables, we can also obtain the maximization of the mutual information,\( \operatorname { I } [ x ^ { t } ; y ^ { t } ] \), as we noted in the main text. We prove this in the following.

First, we obtain an explicit expression of the optimal \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \) in Eq.(8) from the following inequality:

\[ \mathbb { E } \left[ \log \frac { \nu ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } \right] = \sum _ { u ^ { t } , x ^ { t + 1 } } p _ { s } ( y ^ { t } , x ^ { t + 1 } ) \sum _ { x ^ { t } } p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) \log \frac { \nu ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } \leq 0 . \qquad \qquad \qquad ( 1 6 ) \]

The above inequality is derived from the inequality \( F - 1 \geq \log F \) for positive real F, and thus, the optimality condition in Eq.(8) is obtained from the equality, \( F - 1 = \log F \leftrightarrow F = 1 \) with probability 1:

\[ \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) = p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) . \eqno ( 1 7 ) \]

Then, in order to analyze the equality condition of Eq.(3) for \( \nu ( x ^ { t } | x ^ { t + 1 } , y ^ { t } ) \), we calculate the difference \( \Delta \) between the values of −E[Θtν] as calculated with Eqs. (17) and (15), writing

\[ \Delta = \operatorname { E } \left[ \log { \frac { p _ { s } ( x ^ { t } | y ^ { t } , y ^ { t + 1 } , x ^ { t + 1 } ) } { p _ { s } ( x ^ { t } | y ^ { t } , x ^ { t + 1 } ) } } \right] = \operatorname { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] . \eqno ( 1 8 ) \]

Because −E[Θtν] = Ix→y in the case with Eq.(15), we obtain

\[ \begin{array} { r } { \mathrm { I } _ { x \to y } + \mathrm { E } [ \Theta _ { \nu } ^ { t } ] = \mathrm { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] , \eqno ( 1 9 ) } \end{array} \]

in the case with Eq.(17).

Next, we show that the condition

\[ \begin{array} { r } { \mathrm { I } [ y ^ { t + 1 } ; x ^ { t } | x ^ { t + 1 } , y ^ { t } ] = 0 , \eqno ( 2 0 ) } \end{array} \]

is equivalent to the maximization of the mutual information,

\[ \begin{array} { r } { \mathrm { I } [ x ^ { t } ; y ^ { t } ] = \mathrm { H } [ y ^ { t } ] , \eqno ( 2 1 ) } \end{array} \]

on the assumption that the environmental state space Y is not too finely partitioned in comparison with the precision of neural control over the environment. Precisely, we assume that there is no coarse-grained partition \( y ^ { \prime } \) of Y such that the neural control has the same precision on the two partitions Y and \( \mathcal { V } ^ { \prime } \) of the environmental state space. We also assume that \( p _ { s } ( y ^ { t + 1 } | y ^ { t } ) \neq 1 \) for all \( y ^ { t } \) and \( y ^ { t + 1 } \), which holds for most sets of values of the model parameters in a general model. A coarse-grained partition \( y ^ { \prime } \) is a set of subcollections of Y such that \( y ^ { \prime } \cap y ^ { \prime \prime } = \emptyset \) for any \( y ^ { \prime } \neq y ^ { \prime \prime } \in \mathcal { y } ^ { \prime } \) and \( \cup _ { y ^ { \prime } \in \mathcal { y ^ { \prime } } } y ^ { \prime } = \mathcal { y } \). For any such coarse-grained partition \( \mathcal { V } ^ { \prime } \), we require