## 2 PM2.5 Concentration Prediction Approach  
### Deep Belief Network  
Deep belief network (DBN) is a probability generation model with multiple hidden layers. It maximizes the generation probability of entire model by training weights between nodes. Figure 1 shows the basic structure of the DBN. It consists of several restricted Boltzmann machines (RBMs) and a back propagation (BP) neural network. RBM, whose output is fed into the input of the next RBM, is a two-layer neural network with directional connections. Thus, a multi-hidden layer structure can be continuously superimposed. Figure 2 shows the basic structure of the RBM. Wherein \(v_{i}\) and \(h_{i}\) are the visible and hidden nodes of the input and output layers, respectively. The visible and hidden nodes are  
Figure 1: Basic structure of deep belief network (DBN). RBM: restricted Boltzmann machine. directions, while there is no connection between them. \(b\) and \(c\) denote the biases of the output and input layers respectively, while \(w\) denotes the connection weight between the visible node and the hidden node. Thus, the model parameter set \(\theta\) consists of \(b\), \(c\), and \(w\). The energy function of the RBM is defined as  
\[E(v,h)=-\sum_{i=1}^{m}v_{1}c_{i}-\sum_{j=1}^{n}h_{j}b_{j}-\sum_{i=1}^{m}\sum_{j =1}^{n}v_{j}h_{j}w_{ij} \tag{1}\]  
Then, the joint probability distribution function of the visible and hidden nodes can be obtained by the energy function. \[P(v,h|\theta)=\frac{1}{Z(\theta)}e^{-E(v,h|\theta)} \tag{2}\]  
\[Z(\theta)=\sum_{v,h}e^{-E(v,h|\theta)} \tag{3}\]  
where \(Z(\theta)\) is a partition function for normalization. In the RBM, given the input vector \(v\), the activation probability of the hidden node \(h_{i}\) of the output layer can be expressed as  
\[P(h_{i}=1)=\frac{1}{1+\exp(-b_{j}-\sum_{i}v_{i}w_{ij})} \tag{4}\]  
Given the output vector \(h\), the activation probability of the visible node \(v_{i}\) of the input layer can be expressed as  
\[P(v_{i}=1)=\frac{1}{1+\exp(-c_{j}-\sum_{j}h_{j}w_{ji})} \tag{5}\]  
To obtain the optimal solution of the model, the negative log-likelihood function of training set \(D\) is taken as the loss function and is given by  
\[l(\theta,D)=-\frac{1}{N}\sum_{v^{(i)}\in D}\log P(v^{(i)}) \tag{6}\]  
where \(N\) is the size of the training set. After that, each weight is updated by the partial derivatives of the loss function for parameter set \(\theta\), as follows. \[ \begin{array}{c}\frac{\partial\log p(v\theta)}{\partial w_{ij}}=\left<vH_{ ij}\right>_{d}-\left<v_{i}h_{j}\right>_{m}\\ \frac{\partial\log p(v\theta)}{\partial b}=\left<h_{j}\right>_{d}-\left<h_{j} \right>_{m}\\ \frac{\partial\log p(v\theta)}{\partial c}=\left<v_{i}\right>_{d}-\left<v_{i} \right>_{m}\end{array}  \tag{7}\]  
Figure 2: Restricted Boltzmann machine. where \(<:>_{d}\) denotes the statistical probability of the samples, and \(<:>_{m}\) denotes the generation probability of the model. Through adjusting the weights of each node, DBN makes the statistical probability of the samples equal to the generation probability of the model as much as possible. The training process of the DBN is divided into two stages. The first stage is to train each RBM from bottom up. The second stage is to fine-tune the network parameters from top down. In the RBM, the unbiased statistical probability of the samples can be calculated using Equations (4) and (5), while the unbiased generation probability of the model is difficult to obtain. To address this issue, [PERSON] proposed a contrast divergence algorithm [PERSON] (1970) to obtain an approximation of the RBM distribution by one Gibbs sampling. The process can be described mathematically as  
\[ \begin{array}{c}w_{n}\leftarrow\lambda w_{n-1}+\sigma(\left\langle v_{i}h_{ j}\right\rangle^{0}-\left\langle v_{i}h_{j}\right\rangle^{1})\\ b_{n}\leftarrow\lambda b_{n-1}+\sigma(\left\langle vh_{j}\right\rangle^{0}- \left\langle h_{j}\right\rangle^{1})\\ c_{n}\leftarrow\lambda c_{n-1}+\sigma(\left\langle v_{i}\right\rangle^{0}- \left\langle v_{i}\right\rangle^{1})\end{array}