# Learning Algorithm of Boltzmann Machine Based on Spatial Monte Carlo Integration Method
## 4 Comparison of 1-SMCI Learning Method and MPLE
### Comparison from Asymptotic Point of View  
Suppose that we want to approximate the expectation \(\mathrm{E}_{\mathrm{BM}}[x_{i}x_{j}\mid\mathbf{w}]\) in a Boltzmann machine, and assume that the data points are generated independently from the same Boltzmann machine. Here, we re-express \(m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D})\) in Equation (11) and \(m_{ij}^{\mathrm{ISMCI}}(\mathbf{w};\mathcal{D})\) in Equation (23) as  
\[m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D}) =\frac{1}{N}\sum_{\mu=1}^{N}\rho_{ij}^{\mathrm{MPLE}}(\mathbf{x}^{( \mu)},\mathbf{w}), \tag{28}\] \[m_{ij}^{\mathrm{ISMCI}}(\mathbf{w};\mathcal{D}) =\frac{1}{N}\sum_{\mu=1}^{N}\rho_{ij}^{\mathrm{ISMCI}}(\mathbf{x}^{( \mu)},\mathbf{w}), \tag{29}\]  
respectively, where  
\[\rho_{ij}^{\mathrm{MPLE}}(\mathbf{x},\mathbf{w}) :=\frac{1}{2}\big{\{}x_{j}M_{i}(x_{\partial(i)},\mathbf{w})+x_{i}M_{ j}(x_{\partial(j)},\mathbf{w})\big{\}},\] \[\rho_{ij}^{\mathrm{ISMCI}}(\mathbf{x},\mathbf{w}) :=\sum_{x_{i}\in\mathcal{X}}\sum_{x_{j}\in\mathcal{X}}x_{i}x_{j}P _{\mathrm{BM}}(x_{i},x_{j}\mid\mathbf{x}_{N_{1}(\{i,j\})},\mathbf{w}).\]  
Since \(\mathbf{x}^{(\mu)}\)s are the i.i.d. points sampled from \(P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})\), \(\rho_{ij}^{\mathrm{MPLE}}(\mathbf{x}^{(\mu)},\mathbf{w})\) and \(\rho_{ij}^{\mathrm{ISMCI}}(\mathbf{x}^{(\mu)},\mathbf{w})\) can also be regarded as i.i.d. sample points. Thus, \(m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D})\) and \(m_{ij}^{\mathrm{ISMCI}}(\mathbf{w};\mathcal{D})\) are the sample averages over the i.i.d. points. One can easily verify that the two equations \(\sum_{\mathbf{x}}P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})\rho_{ij}^{\mathrm{MPLE}}(\mathbf{x },\mathbf{w})=\mathrm{E}_{\mathrm{BM}}[x_{i}x_{j}\mid\mathbf{w}]\) and \(\sum_{\mathbf{x}}P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})\rho_{ij}^{\mathrm{ISMCI}}(\bm {x},\mathbf{w})=\mathrm{E}_{\mathrm{BM}}[x_{i}x_{j}\mid\mathbf{w}]\) are justified (the former equation can also be justified by using the correlation equality [27]). Therefore, from the law of large numbers, \(m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D})=m_{ij}^{\mathrm{ISMCI}}(\mathbf{w}; \mathcal{D})=\mathrm{E}_{\mathrm{BM}}[x_{i}x_{j}\mid\mathbf{w}]\) in the limit of \(N\to\infty\). This implies that, in the case without model error, the 1-SMCI learning method has the same solution to the MLE in the limit of \(N\to\infty\). From the central limit theorem, the distributions of \(m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D})\) and \(m_{ij}^{\mathrm{ISMCI}}(\mathbf{w};\mathcal{D})\) asymptotically converge to Gaussians with mean \(\mathrm{E}_{\mathrm{BM}}[x_{i}x_{j}\mid\mathbf{w}]\) and variances  
\[v_{ij}^{\mathrm{MPLE}}(\mathbf{w}) :=\frac{1}{N}\Big{(}\sum_{\mathbf{x}}\rho_{ij}^{\mathrm{MPLE}}(\mathbf{x },\mathbf{w})^{2}P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})-\mathrm{E}_{\mathrm{BM}}[x_{i} x_{j}\mid\mathbf{w}]^{2}\Big{)}, \tag{30}\] \[v_{ij}^{\mathrm{ISMCI}}(\mathbf{w}) :=\frac{1}{N}\Big{(}\sum_{\mathbf{x}}\rho_{ij}^{\mathrm{SMCI}}(\mathbf{x },\mathbf{w})^{2}P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})-\mathrm{E}_{\mathrm{BM}}[x_{i} x_{j}\mid\mathbf{w}]^{2}\Big{)}, \tag{31}\]  
respectively, for \(N\gg 1\). For the two variances, we obtain the following theorem. **Theorem 1**. : _For a Boltzmann machine, \(P_{\mathrm{BM}}(\mathbf{x}\mid\mathbf{w})\), defined in Equation (1), the inequality \(v_{ij}^{\mathrm{MPLE}}(\mathbf{w})\geq v_{ij}^{\mathrm{ISMCI}}(\mathbf{w})\) is satisfied for all \((i,j)\in E\) and for any \(N\)._  
The proof of this theorem is given in Appendix A. Theorem 1 states that the variance in the distribution of \(m_{ij}^{\mathrm{MPLE}}(\mathbf{w};\mathcal{D})\) is always larger than (or equal to) that of \(m_{ij}^{\mathrm{ISMCI}}(\mathbf{w};\mathcal{D})\).