## 2 Self-Organizing Deep Neural Network  
### Network Structure  
This paper proposes a self-organizing DNN structure, as shown in Figure 1. The network consists of four parts: input layer, hidden layer, output layer, and instructor signal layer. The first layer is the input layer (visible Layer, V) for receiving the original signal. The \"training of a model\" consists three steps, (1) build an artificial intelligence (AI) model; (2) prepare data to be trained, including model's input and output; (3) use intelligent algorithm to teach the model to output data based on input data. The received signal is then passed to the hidden layer for feature extraction. The hidden layer is usually composed of 2-4 layers of neurons (hidden layer 1-4, L1-L4) depending on the size of the data. The number of neurons in each layer varies. During unsupervised training, V and L1 constitute the first restricted Boltzmann machine (RBM), L1 and L2 constitute the second RBM, and so on. Each RBM consists of a visual layer (input layer) and hidden layer (output layer), and the layers are connected via bi-directional connections (indicated by arrows in Figure 1). There are no connections within the same layer. During the process of unsupervised training, the hidden layer of each RBM is passed the data from the preceding RBM hidden layer and extracts one or more better and more abstract feature expressions from it. Then the feature data is passed to the output layer, which is the third part of the self-organizing DNN. The number of neurons in this layer is usually determined by the task. For example, in classification tasks, the number of neurons is the number of classifications. The function of the instructor signal layer is to provide the target signal for subsequent supervised learning. A conventional DNN is obtained by superimposing several RBMs, and the training process is divided into the unsupervised and supervised parts. Layer-wise pre-training is also used in the unsupervised training process. The RBM training formulae are expressed as:  
\[p\big{(}h_{j}=1\big{)}=\frac{1}{1+\mathrm{e}^{-b_{j}-\sum_{i}v_{i}w_{ij}}} \tag{1}\]  
\[p(v_{i}=1)=\frac{1}{1+\mathrm{e}^{-c_{i}-\sum_{j}h_{j}w_{ji}}} \tag{2}\]  
where \(v_{i}\) is the value of the \(i\)th neuron in the visible layer, \(h_{j}\) is the value of the \(j\)th neuron in the hidden layer, \(b\) and \(c\) are biases of the visible layer and hidden layer, respectively, and \(w_{ij}\) represents the weight between the visible neuron \(i\) and hidden neuron \(j\). Equations (1) and (2) are called the RBM knowledge learning (feature extraction) equation and knowledge inference (data reconstruction) equation, respectively. Let \(\theta=(W,b,c)\). Then, in accordance with the contrastive divergence algorithm, the weight and bias update formulae for unsupervised training are:  
\[\Delta\theta=\left\langle h_{j}^{0}v_{i}^{0}\right\rangle-\left\langle h_{j}^ {1}v_{i}^{1}\right\rangle \tag{3}\]  
\[\theta^{n}=\Delta\theta+\theta^{0} \tag{4}\]  
where \(\left\langle\cdot\right\rangle\) represents the average of the dot product of posterior distribution, \(h_{j}^{0}v_{i}^{0}\) and \(\theta^{0}\) represent the initial state and initial parameters, respectively, \(h_{j}^{1}v_{i}^{1}\) is the product of the visual layer and hidden layer after a Markov chain operation, \(\Delta\theta\) represents the parameter change, and \(\theta^{n}\) is the obtained weight and offset. The self-organizing DNN training process proposed in this paper is divided into two steps. First, each RBM is trained using the greedy layer-wise unsupervised training method, during which adaptive dropout is used to automatically adjust the network structure, thereby decoupling the feature detectors. A regularization enhancement transfer function is designed to introduce the regularization enhancement terms. This term speeds up the process of reducing the number of neural network weights and thereby improves the neural network feature extraction and data reconstruction capabilities.