B.2 Pruning Multiple Edges
Similar to the proof of Lemma 1 , we prove Theorem 2 by induction. Different from Lemma 1 , we induce a slightly different objective:

{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\leq\sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(m)}F_{n}(\bm{x};m)\cdot p_{m}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};m)}{\operatorname{F}_{n}(\bm{x};m)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)}, (7)

where \mathsf{des}(n) is the set of descendent units of n .

Base case: the base case follows directly from the proof of Lemma 1 , and lead to the conclusion in Equation 6 .

Inductive case #1: suppose for all children of a product unit d , Equation 7 holds, we show that Equation 7 also holds for d :

{p}_{d}(\bm{x})-{p}^{\prime}_{d}(\bm{x}) =\prod_{m\in\mathsf{in}(d)}{p}_{m}(\bm{x})-\prod_{m\in\mathsf{in}(d)}{p}^{\prime}_{m}(\bm{x}) =\prod_{m\in\mathsf{in}(d)}{p}_{m}(\bm{x})-\prod_{m\in\mathsf{in}(d)}\Big{(}{p}_{m}(\bm{x})-({p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x}))\Big{)} \leq\sum_{m\in\mathsf{in}(d)}\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x}))\Big{)}\cdot\prod_{m^{\prime}\in\mathsf{in}(d)\backslash m}{p}_{m^{\prime}}(\bm{x}) \overset{(a)}{\leq}\sum_{m\in\mathsf{in}(d)}\sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(m)}F_{n}(\bm{x};d)\cdot p_{d}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};m)}{\operatorname{F}_{n}(\bm{x};m)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)} \leq\sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(d)}F_{n}(\bm{x};d)\cdot p_{d}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};d)}{\operatorname{F}_{n}(\bm{x};d)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)},

where (a) uses the definition that {p}_{d}(\bm{x})=\prod_{m\in\mathsf{in}(d)}{p}_{m}(\bm{x}) .

Inductive case #2: suppose for all children of a sum unit d , Equation 7 holds, we show that Equation 7 also holds for d :

{p}_{d}(\bm{x})-{p}^{\prime}_{d}(\bm{x}) =\sum_{m\in\mathsf{in}(d)\cap(d,m) ot\in\mathcal{E}}\theta_{m|d}\cdot\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\Big{)}+\sum_{m\in\mathsf{in}(d)\cap(d,m)\in\mathcal{E}}\theta_{m|d}\cdot\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\Big{)} \overset{(a)}{=}\sum_{m\in\mathsf{in}(d)\cap(d,m) ot\in\mathcal{E}}\theta_{m|d}\cdot\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\Big{)} \qquad+\sum_{m\in\mathsf{in}(d)\cap(d,m)\in\mathcal{E}}\theta_{m|d}\cdot F_{n}(\bm{x};m)\cdot p_{m}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};m)}{\operatorname{F}_{n}(\bm{x};m)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)},

where (a) follows from the base case of the induction. Next, we focus on the first term of the above equation:

\sum_{m\in\mathsf{in}(d)\cap(d,m) ot\in\mathcal{E}}\theta_{m|d}\cdot\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\Big{)} \leq \sum_{m\in\mathsf{in}(d)\cap(d,m) ot\in\mathcal{E}}\sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(m)}\theta_{m|d}\cdot\Big{(}{p}_{m}(\bm{x})-{p}^{\prime}_{m}(\bm{x})\Big{)} \leq \sum_{m\in\mathsf{in}(d)\cap(d,m) ot\in\mathcal{E}}\sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(m)}\theta_{m|d}\cdot F_{n}(\bm{x};m)\cdot p_{m}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};m)}{\operatorname{F}_{n}(\bm{x};m)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)} \leq \sum_{(n,c)\in\mathcal{E}\cap\mathsf{des}(d)}F_{n}(\bm{x};d)\cdot p_{d}(\bm{x})\cdot\bigg{(}\frac{1}{1-\theta_{c|n}}\frac{\operatorname{F}_{n,c}(\bm{x};d)}{\operatorname{F}_{n}(\bm{x};d)}-\frac{\theta_{c|n}}{1-\theta_{c|n}}\bigg{)},

where the derivation of the last inequality follows from the corresponding steps in the proof of Lemma 1 .

Therefore, from Equation 7 , we can conclude that

\Delta\mathcal{LL}(\mathcal{D},\mathcal{C},\mathcal{E})\leq-\frac{1}{|\mathcal{D}|}\sum_{\bm{x}}\log(1-\sum_{(n,c)\in\mathcal{E}}\operatorname{F}_{n,c}(\bm{x})).

Finally, we prove the approximation step in Equation 3 . Let \epsilon(\cdot)=\sum_{(n,c)\in\mathcal{E}}\operatorname{F}_{n,c}(\cdot)\in[0,1) . We have,

RHS =-\sum_{\bm{x}\in\mathcal{D}}\log(1-\epsilon(\bm{x}))=-\sum_{\bm{x}\in\mathcal{D}}\sum_{k=1}^{\infty}-\frac{\epsilon(\bm{x})^{k}}{k}(Taylor expansion)\leq\sum_{\bm{x}\in\mathcal{D}}\sum_{k=1}^{\infty}\epsilon(\bm{x})^{k} =\sum_{\bm{x}\in\mathcal{D}}\frac{\epsilon(\bm{x})}{1-\epsilon(\bm{x})}=\frac{1}{1-\epsilon}\sum_{\bm{x}\in\mathcal{D}}\epsilon(\bm{x})=\frac{1}{1-\epsilon}\sum_{(n,c)\in\mathcal{E}}\sum_{\bm{x}\in\mathcal{D}}\operatorname{F}_{n,c}(\bm{x})=\frac{1}{1-\epsilon}\sum_{(n,c)\in\mathcal{E}}\operatorname{F}_{n,c}(\mathcal{D}).

∎