# Construction of Air Quality Level Prediction Model Based on STEPDISC-PCA-BP
## 2 Materials and Methods
### Methodology
#### 2.2.3 BP Neural Network  
A BP neural network can train the network model to approximate the functional equation using the input and output vector and get the closest result to the expected output value when the factor vector is input to the functional equation, and its strong nonlinear mapping ability is widely used in classification problems. Therefore, the BP neural network model is chosen to predict the air quality level of the study area. The classical three-layer network structure contains an input layer, a hidden layer, and an output layer, as shown in Figure 2.  
Assuming that the input layer has \(p\) neurons and the received information is a factor vector \(X=(x_{1},\ldots,x_{i},\ldots,x_{p})^{T}\), \((\alpha_{1j},\ldots,\alpha_{ij},\ldots,\alpha_{pj})^{T}\) and \((\omega_{1k},\omega_{2k},\ldots,\omega_{jk},\ldots,\omega_{mk})^{T}\) denote the connection weights of the neurons in each layer. The hidden and output layers, each with \(m\) and \(n\) neurons, receive the dot product of the factor vector and the weights, compare them with the set thresholds \((\beta_{1},\beta_{2},\ldots,\beta_{j},\ldots,\beta_{m})^{T}\) and \((\theta_{1},\ldots,\theta_{k},\ldots,\theta_{n})^{T}\), and substitute them as independent variables into the function \(f\). The function return value is the actual output of the neuron at that layer \((\hat{h}_{1},\hat{h}_{2},\ldots,\hat{h}_{j},\ldots,\hat{h}_{m})^{T}\) (Equation (5)) and \((\hat{y}_{1},\ldots,\hat{y}_{k},\ldots,\hat{y}_{n})^{T}\) (Equation (6)) [14].  
\[\hat{h}_{j}=f(\sum_{i=1}^{p}\alpha_{ij}x_{i}-\beta_{j}) \tag{5}\]  
\[\hat{y}_{k}=f(\sum_{j=1}^{m}\omega_{jk}h_{j}-\theta_{k}) \tag{6}\]  
Figure 2: Three-layer BP neural network structure.  
The Sigmoid is used for the output of the hidden layer which works well when the factor difference is complex (Equation (7)).  
\[f(x)=\frac{1}{1+e^{-x}} \tag{7}\]  
The output layer uses the Softmax which converts the results of each output classification into probability values and selects the category with the highest probability as the final classification. This function is also able to distance the values with large disparities more accurately and classify the results more accurately (Equation (8)).  
\[P(k)=soft\text{max}(y_{k})=\frac{e^{y_{k}}}{\sum\limits_{k=1}^{n}e^{y_{k}}} \tag{8}\]  
The loss function \(E\) is defined as the mean square error between the observed output and the predicted output (Equation (9)).  
\[E=\frac{1}{2}\sum\limits_{j=1}^{n}{(\hat{y}_{j}-y_{j})^{2}} \tag{9}\]  
The network model is trained with the objective of minimizing \(E\). Using the gradient descent algorithm (Equations (10) and (11)), the optimal network model is obtained by iteratively updating the weights \(\omega_{jk}^{(l)}\) and thresholds \(\theta_{k}^{(l)}\) of the neurons in the \(l\)-th layer based on the chain rule starting from \(E\) at the output and propagating backward to the input, and terminating the learning after reaching the expected error or the set number of learning times.  
\[\omega_{jk}^{(l)}=\omega_{jk}^{(l-1)}-b\frac{\partial E}{\partial\omega_{jk}^ {(l-1)}} \tag{10}\]  
\[\theta_{k}^{(l)}=\theta_{k}^{(l-1)}-b\frac{\partial E}{\partial\theta_{k}^{(l -1)}} \tag{11}\]  
In the above equation, \(b\) is called the learning rate, which is too large to fall into a local minimum, and too small to have a long training period for the network model, which usually needs to be adjusted empirically.  
In addition, a reasonable selection of the number of neuron nodes in each layer will also improve the model fitting ability to a great extent. The number of nodes in the input and output layers needs to be determined in conjunction with the actual problem. Too many nodes in the hidden layer are easy to over-fit and weakly generalize the model, while too few nodes are easy to under-fit the model, and in practice, it is often calculated using Equation (12), with \(c\) taking values in the range of 1 to 10 [15].  
\[m=\sqrt{n+p}+c \tag{12}\]