# How Many Bits Of Optimization Can One Bit Of Observation Unlock?

So there’s this thing where a system can perform more bits of optimization on its environment by observing some bits of information from its environment. Conjecture: observing an additional $N$ bits of information can allow a system to perform *at most* $N$ additional bits of optimization. I want a proof or disproof of this conjecture.

I’ll operationalize “bits of optimization” in a similar way to channel capacity, so in more precise information-theoretic language, the conjecture can be stated as: if the sender (but NOT the receiver) observes $N$ bits of information about the noise in a noisy channel, they can use that information to increase the bit-rate by at most $N$ bits per usage.

For once, I’m pretty confident that the operationalization is correct, so this is a concrete math question.

### Toy Example

We have three variables, each one bit: Action ($A$), Observable ($O$), and outcome ($Y$). Our “environment” takes in the action and observable, and spits out the outcome, in this case via an xor function:

$Y = A \oplus O$

We’ll assume the observable bit has a 50/50 distribution. 

If the action is independent of the observable, then the distribution of outcome $Y$ is the same no matter what action is taken: it’s just 50/50. The actions can perform zero bits of optimization; they can’t change the distribution of outcomes at all.

On the other hand, if the actions can be a function of $O$, then we can take either $A = O$ or $A = \bar{O}$ (i.e. not-$O$), in which case $Y$ will be deterministically 0 (if we take $A = O$), or deterministically 1 (for $A = \bar{O}$). So, the actions can apply 1 bit of optimization to $Y$, steering $Y$ deterministically into one half of its state space or the other half. By making the actions $A$ a function of observable $O$, i.e. by “observing 1 bit”, 1 additional bit of optimization can be performed via the actions.

Operationalization
------------------

Operationalizing this problem is surprisingly tricky; at first glance the problem pattern-matches to various standard info-theoretic things, and those pattern-matches turn out to be misleading. (In particular, it’s not just conditional mutual information, since only the sender - not the receiver - observes the observable.) We have to start from relatively basic principles.

The natural starting point is to operationalize “bits of optimization” in a similar way to [info-theoretic channel capacity](https://en.wikipedia.org/wiki/Channel_capacity). We have 4 random variables:

*   “Goal” $G$
*   “Action” $A$
*   “Observable” $O$
*   “Outcome” $Y$

Structurally:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/414452957d813cc2bcc5e088224cbc6d8f34cd7f2e13eb44.png)

(This diagram is a Bayes net; it says that $G$ and $O$ are independent, $A$ is calculated from $G$ and $O$ and maybe some additional noise, and $Y$ is calculated from $A$ and $O$ and maybe some additional noise. So, $P[G, O, A, Y] = P[G] P[O] P[A|G, O] P[Y|A, O]$.) The generalized “channel capacity” is the maximum value of the mutual information $I(G; Y)$, over distributions $P[A|G, O]$.

Intuitive story: the system will be assigned a random goal $G$, and then take actions $A$ (as a function of observations $O$) to steer the outcome $Y$. The “number of bits of optimization” applied to $Y$ is the amount of information one could gain about the goal $G$ by observing the outcome $Y$.

In information theoretic language:

*   $G$ is the original message to be sent
*   $A$ is the encoded message sent in to the channel
*   $O$ is noise on the channel
*   $Y$ is the output of the channel

Then the generalized “channel capacity” is found by choosing the encoding $P[A|G, O]$ to maximize $I(G; Y)$.

I’ll also import one more assumption from the standard info-theoretic setup: $G$ is represented as an arbitrarily long string of independent 50/50 bits.

So, fully written out, the conjecture says:

> Let $G$ be an arbitrarily long string of independent 50/50 bits. Let $A$, $O$, and $Y$ be finite random variables satisfying
> 
> $P[G, O, A, Y] = P[G] P[O] P[A|G, O] P[Y|A, O]$
> 
> and define
> 
> $\Delta := (\max_{P[A|G, O]} \; I(G;Y)) - (\max_{P[A|G, O]} \; I(G;Y) \; \text{subject to} \; P[A|G, O] = P[A|G])$
> 
> Then
> 
> $\Delta \leq H(O)$

Also, one slightly stronger bonus conjecture: $\Delta$ is at most $I(A; O)$ under the unconstrained maximal $P[A|G, O]$.

(Feel free to give answers that are only partial progress, and use this space to think out loud. I will also post some partial progress below. Also, thankyou to Alex Mennen for some help with a couple conjectures along the path to formulating this one.)