# Orthogonal Neural Network: An Analytical Model for Deep Learning
## 1 Introduction  
Artificial intelligence technology has been used to achieve unquestionable success in various fields, bringing a new way of thinking to human science, technology, and productive life. As one of the most crucial technical methods of artificial intelligence, deep learning is the direction of research experiencing the most attention in academia and industry today. Deep learning, with its end-to-end learning approach, is capable of efficient automatic feature extraction and has experienced substantial improvements in accuracy, triggering yet another peak in AI development. However, deep learning models are characterized by complex models, a vast number of parameters, and nonlinearities, somewhat resembling a black box, and meaningful insight into their inner working mechanisms is lacking. Deep neural networks have been shown to have low robustness, e.g., in studies of adversarial attacks on convolutional neural networks [1; 2; 3; 4; 5; 6] and graph neural networks [7; 8; 9; 10]. As a result, deep neural networks are limited in their use in some areas where security is a concern. In certain research advances in explainable AI [11], some extremely clever research perspectives on deep learning models have been proposed for analysis. For example, the CAM [12] algorithm can be used to locate the main regions of interest in image classification models. The LIME [13] algorithm uses the idea of local linearization to analyze linearization within the neighborhood of results. In this paper, we propose an extended model of the deep neural networks that can analyze the results from each step of computation in the deep neural networks and decompose them into the parts corresponding to each input feature. Analysis is a primary method for studying a problem. The analysis of the object as a whole is formed by decomposing the object of study into its parts and synthesizing themafter studying each part separately. Therefore, how to break down the research object from the whole into its parts is an important starting point. It is envisaged that a microscopic perspective can be chosen as a starting point, where a single input numerical feature of a deep learning model--for example, a pixel in image data or a single feature on a node in graph data--serves as the unit of analysis. Related studies in the literature have not usually been conducted from the perspective of individual input features. There are two main reasons for this. First, the number of features and model parameters is often huge, and it is difficult to intuitively focus the research perspective on a single input feature. Furthermore, the basic idea of deep learning is to train the model in a large amount of feature data to obtain the model parameters with the minimum loss, and in this perspective specifically focusing on individual features is of little significance. In human exploration of the laws of nature, many superposition phenomena have been discovered: for example, waves in the water and the superposition of different light colors. By studying the decomposition of waves based on wavelength, it is possible to determine the pattern of wave motion. Research on the decomposition of light based on wavelength is precisely spectral theory. In [PERSON]'s classic work on heat conduction, _The Analytical Theory of Heat_[14], the problem of triangular series expansion is studied and discussed. After continuous refinement by mathematicians, the Fourier series, as it has come to be known, was further developed and refined in the study of signal analysis and processing as the Fourier transform. Since trigonometric functions have a clear physical meaning in the study of signals, expanding the signal into a Fourier series representation is equivalent to decomposing the signal into different frequencies for study. It is also inferred that the signal is decomposed into frequency space. The information loaded on each frequency is irrelevant, because the trigonometric function representing frequency is orthogonal.