Theorem: Conditional Entropy of Join as Sum

Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.
Let $\AA, \CC, \DD \subseteq \Sigma$ be finite sub-$\sigma$-algebras.
Then:
:$\ds \map H {\AA \vee \CC \mid \DD} = \map H {\AA \mid \DD} + \map H {\CC \mid \AA \vee \DD} $
where:
:$\map H {\cdot \mid \cdot}$ denotes the conditional entropy
:$\vee$ denotes the join

Proof:
Consider the generated finite partitions:
:$\xi := \map \xi \AA$
:$\eta := \map \xi \CC$
:$\gamma := \map \xi \DD$
By definition of conditional entropy of finite sub-sigma-algebra, we shall show:
:$\map H {\xi \vee \eta \mid \gamma} = \map H {\xi \mid \gamma} + \map H {\eta \mid \xi \vee \gamma}$
Then:
{{begin-eqn}}
{{eqn | l = \map H {\xi \vee \eta \mid \gamma}
      | r = \sum_{\substack {D \mathop \in \gamma \\ \map \Pr D \mathop > 0} } \map \Pr D \sum_{B \mathop \in \xi \vee \eta} \map \phi {\dfrac {\map \Pr {B \cap D} } {\map \Pr D} }
      | c = {{Defof|Conditional Entropy of Finite Partitions}}
}}
{{eqn | r = \sum_{\substack {D \mathop \in \gamma \\ \map \Pr D \mathop > 0} } \map \Pr D \sum_{\substack {A \mathop \in \xi \\ C \mathop \in \eta} } \map \phi {\dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr D} }
      | c = {{Defof|Join of Finite Partitions}}
}}
{{eqn | r = - \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap C \cap D} \map \log {\dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr D} }
      | c = Definition of $\phi$
}}
{{eqn | r = - \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap C \cap D} \map \log { \dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr {A \cap D} } \dfrac {\map \Pr {A \cap D} } {\map \Pr D } } 
}}
{{eqn | r = - \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap C \cap D} \map \log { \dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr {A \cap D} } }
}}
{{eqn | r = \sum_{\substack {\tuple {A, C, D} \mathop \in \xi\times\eta\times\gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap C \cap D} \map \log { \dfrac {\map \Pr {A \cap D} } {\map \Pr D } }
      | o =
      | ro = -
      | c = Real Logarithm is Completely Additive
}}
{{eqn | r = \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap D} \map \phi { \dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr {A \cap D} } }
}}
{{eqn | r = \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \dfrac {\map \Pr {A \cap C \cap D} \map \Pr D} {\map \Pr {A \cap D} } \map \phi { \dfrac {\map \Pr {A \cap D} } {\map \Pr D } }
      | o =
      | ro = +
      | c = Definition of $\phi$
}}
{{eqn | r = L + R
      | o = =:
}}
{{end-eqn}}
Now:
{{begin-eqn}}
{{eqn | l = L
      | r = \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \map \Pr {A \cap D} \map \phi { \dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr {A \cap D} } }
}}
{{eqn | r = \sum_{\substack {\tuple {A, D} \mathop \in \xi \times \gamma \\ \map \Pr {A \cap D} > 0 } } \map \Pr {A \cap D} \sum_{C \in \eta} \map \phi { \dfrac {\map \Pr {A \cap C \cap D} } {\map \Pr {A \cap D} } }
}}
{{eqn | r = \sum_{\substack {F \mathop \in \xi \vee \gamma \\ \map \Pr F > 0 } } \map \Pr F \sum_{C \mathop \in \eta} \map \phi { \dfrac {\map \Pr {C \cap F} } {\map \Pr F } }
      | c = {{Defof|Join of Finite Partitions}}
}}
{{eqn | r = \map H {\eta \mid \xi \vee \gamma}
      | c = {{Defof|Conditional Entropy of Finite Partitions}}
}}
{{end-eqn}}
and:
{{begin-eqn}}
{{eqn | l = R
      | r = \sum_{\substack {\tuple {A, C, D} \mathop \in \xi \times \eta \times \gamma \\ \map \Pr {A \cap C \cap D} > 0 } } \dfrac {\map \Pr {A \cap C \cap D} \map \Pr D} {\map \Pr {A \cap D} } \map \phi { \dfrac {\map \Pr {A \cap D} } {\map \Pr D } }
}}
{{eqn | r = \sum_{\substack {\tuple {A, D} \mathop \in \xi \times \gamma \\ \map \Pr {A \cap D} > 0 } } \sum_{C \in \eta} \dfrac {\map \Pr {A \cap C \cap D} \map \Pr D} {\map \Pr {A \cap D} } \map \phi { \dfrac {\map \Pr {A \cap D} } {\map \Pr D } }
}}
{{eqn | r = \sum_{\substack {A \mathop \in \xi \\ \map \Pr D > 0 } } \map \Pr D \sum_{C \mathop \in \eta} \map \phi { \dfrac {\map \Pr {A \cap D} } {\map \Pr D } }
      | c = $\ds \sum _{C \in \eta} \map \Pr {A \cap C \cap D} = \map \Pr {A \cap D}$
}}
{{eqn | r = \map H {\xi \mid \gamma}
      | c = {{Defof|Conditional Entropy of Finite Partitions}}
}}
{{end-eqn}}
{{qed}}