# Research on Generalized Hybrid Probability Convolutional Neural Network
## 3 SOPNN Classifier in the GHP-CNN
### The Hidden Parameter Optimization in the SOPNN Using the EM Algorithm  
The EM algorithm solves the parameter estimation problem using iterative methods in various probability models. Each iteration consists of two steps, called the \(\mathbf{E}\) step and the \(\mathbf{M}\) step. The \(\mathbf{E}\) step uses the existing estimated parameters and the observed data to calculate the expectation of the hidden parameters. The \(\mathbf{M}\) step uses the expectation obtained in the \(\mathbf{E}\) step and maximum likelihood estimation to estimate the hidden parameter. Let a set \(X\) that is used for training the SOPNN contain \(L\) samples from \(c\) classes, and let the \(j\)th class contain \(L_{j}\) training samples. That is, \(X=\left\{\mathbf{x}^{l}\right\}_{l=1}^{L}=\left\{\left\{\mathbf{x}^{lj} \right\}_{l=1}^{L_{j}}\right\}_{j=1}^{c}.\) With a hyperspherical crown mapping, \(X\) can be projected as \(\hat{X}=\left\{\hat{\mathbf{x}}^{l}\right\}_{l=1}^{L}=\left\{\left\{\hat{ \mathbf{x}}^{lj}\right\}_{l=1}^{L_{j}}\right\}_{j=1}^{c}.\) According to the independence of the sample, the corresponding log-likelihood function can be obtained:  
\[Log=\sum_{j=1}^{c}\sum_{l=1}^{L_{j}}\log y_{j}\left(\hat{\mathbf{x}}^{lj} \right)=\sum_{j=1}^{c}\sum_{l=1}^{L_{j}}\log\sum_{m=1}^{M_{j}}\lambda_{mj} \phi_{mj}\left(\hat{\mathbf{x}}^{lj}\right) \tag{5}\]  
Figure 4: The overall structure of the SOPNN classifier. Using the EM algorithm to solve for the maximum value of the likelihood function, the following iterative formulas can be obtained:  
\[\hat{\mathbf{u}}^{mj}=\frac{\sum_{l=1}^{L_{j}}w_{mj}\left(\hat{\mathbf{x}}^{lj} \right)\hat{\mathbf{x}}^{lj}}{\sum_{l=1}^{L_{j}}w_{mj}\left(\hat{\mathbf{x}}^{lj }\right)} \tag{6}\]  
\[\sigma_{mj}^{2}=\frac{\sum_{l=1}^{L_{j}}w_{mj}\left(\hat{\mathbf{x}}^{lj} \right)\left(\hat{\mathbf{x}}^{lj}-\hat{\mathbf{u}}^{mj}\right)^{T}\left(\hat{ \mathbf{x}}^{lj}-\hat{\mathbf{u}}^{mj}\right)}{\left(n+1\right)\sum_{l=1}^{L_ {j}}w_{mj}\left(\hat{\mathbf{x}}^{lj}\right)} \tag{7}\]  
\[\lambda_{mj}=\frac{\sum_{l=1}^{L_{j}}w_{mj}\left(\hat{\mathbf{x}}^{lj}\right)} {L_{j}} \tag{8}\]  
\[w_{mj}\left(\hat{\mathbf{x}}^{lj}\right)=\frac{\lambda_{mj}\phi_{mj}\left( \hat{\mathbf{x}}^{lj}\right)}{\sum_{m=1}^{M_{j}}\lambda_{mj}\phi_{mj}\left( \hat{\mathbf{x}}^{lj}\right)} \tag{9}\]  
If the initial pattern node and kernel width are known, we can obtain the corresponding parameter value \(\lambda_{mj}\) in Equation (4) by using the above iterative formulas. The termination criterion of the EM algorithm is that the absolute value of the difference between the hidden parameter \(\theta^{(i+1)}\) of the \((i+1)\)th order and the hidden parameter \(\theta^{(i)}\) of the \((i)\)th order is less than a given positive \(\epsilon\), where \(\epsilon=0.01\). There are three differences between the used SOPNN classifier in this paper and the traditional probability neural network (PNN) classifier. First, the hyperspherical crown normalization method in the SOPNN is a map without separable information loss, and the hypersphere normalization in a traditional PNN is a map with separable information loss. The conclusion that the hyperspherical crown normalization is superior to the traditional hypersphere normalization in terms of separability was proven in [29]. Second, when the training set is large, the number of pattern nodes in the traditional PNN is equal to the number of training samples, which leads to a complex network structure. In this paper, the kernel coverage method based on the potential function in [30] is used to adaptively generate pattern nodes, which makes the number of pattern nodes far fewer than the number of training samples. This ensures that the network structure of the SOPNN is simpler than that of the traditional PNN. Third, the SOPNN assigns a local intraclass weight coefficient to each pattern node, which reflects the prior distribution information of the local region of the pattern node.