Theorem: Conditional Entropy Decreases if More Given

:$\CC \subseteq \DD \implies \map H {\AA \mid \CC} \ge \map H {\AA \mid \DD}$

Proof:
Consider the generated finite partitions:
:$\xi := \map \xi \AA$
:$\eta := \map \xi \CC$
:$\gamma := \map \xi \DD$
By Generating Partition Preserves Order, $\CC \subseteq \DD$ implies:
:$\eta \le \gamma$
where $\le$ denotes the refinement.
By {{Defof|Conditional Entropy of Finite Sub-Sigma-Algebra}}, we shall show:
:$\map H {\xi \mid \eta} \ge \map H {\xi \mid \gamma}$
Recall that $\eta \le \gamma$ implies:
:$\forall B \in \eta, \forall D \in \gamma \implies D \subseteq B \; \text{or} \; B \cap D = \O$
In particular, we have:
:$(1): \quad \forall B \in \eta, \forall D \in \gamma: \map \Pr {B \cap D} = \begin{cases} \map \Pr D &: D \subseteq B \\ 0 &: B \cap D = \O \end{cases}$
Thus, for all $A \in \xi$ and $B \in \gamma$ such that $\map \Pr B > 0$:
{{begin-eqn}}
{{eqn | l = \dfrac {\map \Pr {A \cap B} } {\map \Pr B}
      | r = \dfrac 1 {\map \Pr B} \sum_{\substack {D \mathop \in \gamma \\ D \mathop \subseteq B} } \map \Pr {A \cap D}
}}
{{eqn | r = \dfrac 1 {\map \Pr B} \sum_{\substack {D \mathop \in \gamma \\ \map \Pr D > 0} } \dfrac {\map \Pr {B \cap D} } {\map \Pr D} \map \Pr {A \cap D}
      | c = apply $(1)$
}}
{{eqn | r = \sum_{\substack {D \mathop \in \gamma \\ \map \Pr D > 0} } \dfrac {\map \Pr {B \cap D} } {\map \Pr B} \dfrac {\map \Pr {A \cap D} } {\map \Pr D}
}}
{{end-eqn}}
The function $\phi$ used in {{Defof|Conditional Entropy of Finite Partitions}} is concave since the second derivative is negative:
:$\forall x > 0 : \map {\phi ' '} x = -\dfrac 1 x < 0$
Note that the concavity holds including $x = 0$, since $\phi$ is continuous there.
So we obtain:
{{begin-eqn}}
{{eqn | n = 2
      | l = \map \phi {\dfrac {\map \Pr {A \cap B} } {\map \Pr B} }
      | r = \sum_{\substack {D \mathop \in \gamma \\ \map \Pr D > 0} } \dfrac {\map \Pr {B \cap D} } {\map \Pr B}  \map \phi {\dfrac {\map \Pr {A \cap D} } {\map \Pr D} }
      | o = \ge
      | c = Jensen's inequality
}}
{{eqn | r = \sum_{\substack {D \mathop \in \gamma \\ D \mathop \subseteq B \\ \map\Pr D > 0} } \dfrac {\map \Pr D} {\map \Pr B}  \map \phi {\dfrac {\map \Pr {A \cap D} } {\map \Pr D} }
      | c = apply $(1)$
}}
{{end-eqn}}
Therefore:
{{begin-eqn}}
{{eqn | l = \map H {\xi \mid \eta}
      | r = \sum_{\substack {B \mathop \in \eta \\ \map \Pr B \mathop > 0} } \map \Pr B \sum_{A \mathop \in \xi} \map \phi {\dfrac {\map \Pr {A \cap B} } {\map \Pr B} }
      | c = {{Defof|Conditional Entropy of Finite Partitions}}
}}
{{eqn | r = \sum_{\substack {B \mathop \in \eta \\ \map \Pr B \mathop > 0} } \map \Pr B \sum_{A \mathop \in \xi} \sum_{\substack {D \mathop \in \gamma \\ D \mathop \subseteq B \\ \map\Pr D > 0} } \dfrac {\map \Pr D} {\map \Pr B}  \map \phi {\dfrac {\map \Pr {A \cap D} } {\map \Pr D} }
     | o = \ge
     | c = substituting from $(2)$
}}
{{eqn | r = \sum_{\substack {B \mathop \in \eta \\ \map \Pr B \mathop > 0} } \sum_{\substack {D \mathop \in \gamma \\ D \mathop \subseteq B \\ \map\Pr D > 0} } \sum _{A \mathop \in \xi} \map \Pr D  \map \phi {\dfrac {\map \Pr {A \cap D} } {\map \Pr D} }
}}
{{eqn | r = \sum_{\substack {D \mathop \in \gamma \\ \map\Pr D > 0} } \sum _{A \mathop \in \xi} \map \Pr D  \map \phi {\dfrac {\map \Pr {A \cap D} } {\map \Pr D} }
      | c = since $\eta \le \gamma$
}}
{{eqn | r = \map H {\xi \mid \gamma}
      | c = {{Defof|Conditional Entropy of Finite Partitions}}
}}
{{end-eqn}}
{{qed}}