# Orthogonal Neural Network: An Analytical Model for Deep Learning
## 3 ONN Model Applied on Convolutional Neural Networks
### Discussion  
The features enter the first four layers from the input layer and are computed according to the orthogonal neural network operation rules based on the extended operation rules. The output of the fourth layer is an \(E^{12\times 4\times 4}\) matrix, which is spread to an \(E^{192}\) matrix and then enters the fully connected network, and the final output is an \(E^{10}\) matrix. This is further expressed as \(E^{10}=\{\mathbf{e}_{1},\mathbf{e}_{2},\cdots,\mathbf{e}_{10}\}\), so \(\{S(\mathbf{e}_{1}),S(\mathbf{e}_{2}),\cdots,S(\mathbf{e}_{10})\}\), which corresponds to the output of the original network model. However, each \(\mathbf{e}\in E\) in the computation process contains a lot of useful information, based on which a wealth of analysis can be performed on the convolutional neural network model. Since \(\mathbf{e}\in\{\sum_{i}c_{i}\boldsymbol{\alpha}_{i},c_{i}\in\mathbb{R},i\in \mathbb{N},\boldsymbol{\alpha}_{i}\in\mathcal{A}\}\), in this case, \(i\in\{i\in\mathbb{N},1\leq i\leq 784\}\), a corresponding \(\boldsymbol{\alpha}_{i}\) is assigned to each pixel point in the modulation stage of the input layer, and the value \(c_{i}\) on \(\boldsymbol{\alpha}_{i}\) in the computation process is the result from computation of the pixel point corresponding to the input layer in the convolutional neural network. All matrices with elements of \(\mathbf{e}\in E\) generated in the computation process of the convolutional neural network contain a linear combination of its relationship with the value of each pixel in the input layer. Figure 2a is an original image in the dataset for an Ankle Boot, corresponding to an Ankle Boot label of 10. This is equivalent to the result of calculating \(\{S(\mathbf{e}_{1}),S(\mathbf{e}_{2}),\cdots,S(\mathbf{e}_{10})\}\) in the vector matrix whose output is \(E^{10}=\{\mathbf{e}_{1},\mathbf{e}_{2},\cdots,\mathbf{e}_{10}\}\), so \(S(\mathbf{e}_{10})\) is the expected result from calculation of the maximum. The image is input into the orthogonal neural network, where the network parameters are the same as for the original convolution neural network, and the correct classification expectation is obtained, that is, maximum of \(S(\mathbf{e}_{10})\). Further, we put \(\mathbf{e}_{10}\) through the correspondence between the orthogonal vectors and the initial features to recover the position of each component of the result in the original image, as shown in Figure 2b, where blue is the component with negative values and red is the component with positive values, and the darker the color, the larger the value. It can be observed from the figure that the initial pixel points have positive and negative effects on the calculation. For convenient observation, the figure uses a gray background to describe the outline of the original image content. The feature analysis plots corresponding to Coat and T-shirt for the categories in the calculation are shown in Figure 2c,d. The weights in the corresponding classification calculation results of the pixel points drawn in the figure show no obvious distribution pattern in the overall view. However, from the numerical point of view, the relationship between input pixels and output calculation results is established, and the calculation basis for image classification by the convolutional neural network can be obtained clearly and unambiguously, breaking the black box of the deep learning model and establishing the model with pixel points as the analysis angle. Moreover, it can be observed from the feature analysis graph that both positive and negative numbers are included in the classification calculation result, which is the result obtained by combining multiple features, reflecting the complexity of the deep learning model. Both positive and negative numbers play an essential role in these results, with  
Figure 2: Original and analysis samples: (**a**) original image; (**b**) analysis of Ankle Boot classification; (**c**) analysis of Coat classification; (**d**) analysis of T-shirt classification.