## 2 Self-Organizing Deep Neural Network
### Supervised Training Algorithm  
The backpropagation (BP) algorithm is used in supervised training of the self-organizing DNN. The last layer of the neural network receives the output of the last RBM. The difference between the actual signal and the network output signal produces a feedback error. The error is used to perform local adjustments of the overall weights of the neural network. The process is divided into two steps: First, the self-organizing DNN is expanded into a forward network and the data signal is transmitted to the input layer, then to the next layer, and so on until the output layer; second, the backpropagation error is generated for fine tuning the neural network parameters. The specific steps are as follows:  
1. Initialize the forward neural network, set the step length to \(K\).
2. Calculate the forward output, \[y_{j}(l)=\sum w_{ij}y_{j}(l-1)\] (17) where \(y_{j}(l)\) is the output of the \(j\)th neuron of the \(l\)th layer.
3. Calculate the error signal based on: \[e_{j}=R_{j}-Y_{j}\] (18) where \(R\) represents the teacher's signal, which represents ideal predicted temperature, and it is taken by hand when the whole dataset is processed in experiment. \(Y\) is the value of output layer, and \(e\) the error signal.
4. Generate the backpropagation error. For neurons in the output layer, this is: \[\delta_{j}(l)=e_{j}Y_{j}\Big{[}1-Y_{j}\Big{]}\] (19) For neurons in the hidden layer, we have: \[\delta_{j}(l-1)=y_{j}(l)\Big{[}1-y_{j}(l)\Big{]}\sum\delta_{j}(l)w_{ij}(l)\] (20)
5. Modify the weights based on: \[w_{ij}(l)=w_{ij}(l)+\eta\delta_{j}y_{j}(l)\] (21) where \(\eta\) is the learning rate.
6. When the iteration reaches \(K\), terminate the loop; otherwise, return to Step (2).  
During unsupervised training, the regularization enhancement factor does not change the calculation processes of the visible layer and hidden layer, as shown in Equation (7). Instead, the weights of the network are adjusted only by affecting the shares of the two parts of the new objective function. Moreover, the value range of the neural network unit does not change (0 and 1). The combined effect of the above factors is that unsupervised process of self-organizing DNN must also converge under the same condition as the conventional unsupervised process. The conventional BP algorithm is used during supervised training. In accordance with the Weierstrass approximation theorem [23], the supervised process must converge.  
In this section, the detailed training processes (including unsupervised and supervised training) and the improvements (unsupervised-dropout and unsupervised-regularization) have been fully introduced. In next section, the improved model will be proposed as an alternative to thermal error modeling, because (1) DNN, proposed as a big data technology, has been proved great successes on many feature learning problems; (2) the process of unsupervised learning has been improved in two ways, which have greatly increased DNN's feature learning and regression ability; (3) to solve thermal error modeling, a big data method is required because feature learning is very important in this process, besides, thermal error data is used as input and temperature data is used as output, which is a classic regression problem.