## Appendix B Additional Examples
Since \(X_{1}\) and \(X_{2}\) have no parents, their entropy components simply sum over their marginal distributions:_  
\[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]3 \[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]4  
_For \(X_{3}\),_  
\[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]5  
_where_  
\[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]6 \[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]7 \[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]8 \[X_{5} =0.2+\varepsilon x_{5_{6}}, \varepsilon x_{5_{6}} \sim N(0,0.36);\]9  
_and where (multiplying the marginal probabilities for \(X_{1}\) and \(X_{2}\), which are marginally independent)_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]0 \[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]1  
_giving_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]2_Finally, for \(X_{4}\)_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]3  
_where_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]4 \[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]5  
_and \(\mathrm{P}(X_{3}=e)=0.601\), \(\mathrm{P}(X_{3}=f)=0.399\), giving_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]6  
_Combining all these figures, we obtain \(\mathrm{H}(\mathcal{B})\) as_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]7  
_as before._  
_In general, we would have to compute the probabilities of the parent configurations of each node using a junction tree as follows:_  
1. _We construct the moral graph of_ \(\mathcal{B}\)_, which contains the same arcs (but undirected) as its DAG plus_ \(X_{1}\) _--_ \(X_{2}\)_._
2. _We identify two cliques_ \(\mathrm{C}_{1}=\{X_{1},X_{2},X_{3}\}\) _and_ \(\mathrm{C}_{2}=\{X_{3},X_{4}\}\) _and a separator_ \(S_{12}=\{X_{3}\}\)_._
3. _We connect them to create the junction tree_ \(\mathrm{C}_{1}\) _--_ \(S_{12}\) _--_ \(\mathrm{C}_{2}\)_._
4. _We initialise the cliques with the respective distributions_ \(\mathrm{P}(\mathrm{C}_{1})=\mathrm{P}(X_{1},X_{2},X_{3})\)_,_ \(\mathrm{P}(\mathrm{C}_{2})=\mathrm{P}(X_{3},X_{4})\) _and_ \(\mathrm{P}(S_{12})=\mathrm{P}(X_{3})\)_._
5. _We compute_ \(\mathrm{P}(X_{1},X_{2})=\sum_{x_{3}\in\{e,f\}}\mathrm{P}(\mathrm{C}_{1})\) _and_ \(\mathrm{P}(X_{3})=\mathrm{P}(S_{12})\)_._  
**Example A4** (Entropy of a GBN). : _Consider the GBN \(\mathcal{B}\) from Figure 1 top, whose global distribution we derived in Example 2. If we plug its covariance matrix \(\Sigma_{\mathcal{B}}\) into the entropy formula for the multivariate normal distribution we obtain_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]8  
_Equivalently, plugging the \(\sigma^{2}_{X_{i}}(\mathcal{B})\) into (12) we have_  
\[X_{6} =0.1+0.2X_{4}+\varepsilon x_{5_{6}}, \varepsilon x_{6} \sim N(0,1).\]9  
**Example A5** (KL between GBNs with parameters estimated from data). : _Consider the DAGs for the BNs \(\mathcal{B}\) and \(\mathcal{B}^{\prime}\) and the 10 observations shown in Figure A1.