# Research on Generalized Hybrid Probability Convolutional Neural Network
## 3 SOPNN Classifier in the GHP-CNN  
### The Overall Structure of the SOPNN Classifier  
As shown in Figure 4, the SOPNN classifier consists of four layers. The first layer is the input layer for receiving the input feature vectors. A traditional PNN adopts a unit hypersphere normalization for the pattern vectors and input samples, which ensures that the distance between different vectors in a network can be expressed in a simple inner product form. That is, the squared distance between \(\mathbf{x}\) and \(\mathbf{y}\) can be expressed as \(\|\mathbf{x}-\mathbf{y}\|=2-2\mathbf{x}^{T}\mathbf{y}\) if \(\mathbf{x}\) and \(\mathbf{y}\) are normalized. Note that such a simple inner form can be easily realized with a neuron structure. However, with a hypersphere normalization, all sample points that are collinear with the sphere center overlap. To avoid the separability problem in the hypersphere normalization process, the SOPNN used the unit hyperspherical crown normalization \(\widehat{\mathbf{x}}=\widehat{\mathbf{x}}/\|\mathbf{\hat{x}}\|\) from reference [29], where \(\widehat{\mathbf{x}}=[\mathbf{x},h]\in\mathbf{R}^{n+1}\) and \(\mathbf{x}\) are n-dimensional features of the target image extracted by the CFM. \(h\) is the additional dimension.  
The second layer is the pattern layer, and the pattern nodes in this layer receives the vector \(\mathbf{\hat{x}}\) from the input layer. Let a pattern node \(\mathbf{\hat{u}}^{mj}\) be the \(m\)th center in the \(j\)th class ( \(1\leq j\leq c,\quad 1\leq m\leq M_{j}\)). Its output can be given by  
\[\phi_{mj}(\mathbf{\hat{x}})=\frac{\exp\left[-\frac{\left(\mathbf{\hat{x}}- \mathbf{\hat{u}}^{mj}\right)^{T}\left(\mathbf{\hat{x}}-\mathbf{\hat{u}}^{mj} \right)}{2\left(\sigma_{mj}\right)^{2}}\right]}{\left(2\pi\right)^{(n+1)/2} \left(\sigma_{mj}\right)^{(n+1)}} \tag{2}\]  
where \(\sigma_{mj}\) denotes the kernel width, \(M_{j}\) denotes the number of pattern nodes in the \(j\)th class and the total number of pattern nodes is \(M\). That is, \(M=\sum_{j=1}^{c}M_{j}\).  
The third layer is the summation layer, which includes \(c\) neurons and one neuron per class. Each neuron calculates a conditional probability density function \(y_{j}(\mathbf{\hat{x}})\) for each class using the following weighted summation form.  
\[y_{j}(\mathbf{\hat{x}})=\sum_{m=1}^{M_{j}}\lambda_{mj}\phi_{mj}(\mathbf{\hat{ x}}),\quad 1\leq j\leq c \tag{3}\]  
Figure 3: The accuracy increase of different \(t\) unknown subclasses compared with the original accuracy.  
where \(\lambda_{mj}\) denotes the prior distribution information of the training samples of the \(m\)th center in the \(j\)th class and satisfies \(\sum_{m=1}^{M_{j}}\lambda_{mj}=1(1\leq j\leq c)\).  
The fourth layer is the output layer, which makes decisions according to the following equation.  
\[C^{*}(\mathbf{x})=\arg\max\{p_{j}y_{j}(\mathbf{\hat{x}})\},j=1,2,\cdots,c \tag{4}\]  
where \(c\) is the class number and \(C^{*}(\mathbf{x})\) is the predicted class of \(\mathbf{x}\). \(p_{j}\) denotes the prior probability of the \(j\)th class. Obviously, all the parameters have an interpretable physical meaning in our classifier.