# Visual Analysis of Spatiotemporal Data Predictions with Deep Learning Models
## 5 System Evaluation with Air Pollutant Prediction Models  
In this section, we describe the deep learning modeling process within the proposed system, using spatiotemporal air pollutant data. The deep learning modeling process involves selecting features, time lags, and deep learning algorithms, according to the correlation information between variables, temporal autocorrelation, and spatial autocorrelation. The spatial autocorrelation is computed with Moran's I [52] and the local indicator of spatial association (LISA [53]. Moran's I is one of the representative statistics for testing global spatial autocorrelation, confirming whether the values of specific variables in the analysis target region are correlated. Moran's I indicates how similarly the values of the variables measured in adjacent spaces are distributed. When the value of Moran's I is close to 1, the adjacent neighboring spatial units have similar values, and when the value of Moran's I is close to \(-1\), the neighboring spatial units have different values. LISA (local indicator of spatial association) is sometimes called local Moran's I because it shows local spatial dependence. LISA makes it possible to identify the occurrence of local clustering patterns of a given variable in space. The proposed visualization system supports the deep learning modeling of spatiotemporal data by visualizing the information and prediction results required for better modeling. Therefore, the system enables us to observe the prediction results of the deep learning model to discover problems within the modeling.  
The purpose of deep learning modeling with the air pollutant data introduced in Section 3 is to predict the amount of air pollution in the future. In this paper, we train PM\({}_{2.5}\) with the temporal and spatiotemporal predictions of deep learning models. Then, we calculate the mean absolute percentage error (MAPE) from the test data set not used for the training as a measure of the performance of the model. The predicted values by the deep learning model are inserted into the interpolation algorithm. The interpolated continuous results are projected on a map, which makes it easy to recognize the visual distribution of the prediction.  
Our spatiotemporal data prediction modeling system, as shown in Figure 1, is a web-based application developed under the Flask framework, and visualization modules are implemented using D3.js. In the back-end, the prediction network models, such as LSTM, GRU, and Convolutional LSTM, are implemented with Python. Figure 1 presents our air pollutant prediction modeling system that enables us to compare spatiotemporal data prediction models and investigate the prediction performance. In Figure 1a, the scatterplot shows the correlation and probability distribution between input variables. We compare five input variables to capture the correlations and data distributions and observe that PM\({}_{2.5}\) and PM\({}_{10}\) are highly correlated. The system also presents spatial autocorrelation (Moran's I) in (b), where LISA is visualized. We recognize high-high and low-low LISA as clusters. The temporal autocorrelation is plotted in (c). We recognize that the temporal autocorrelation of PM\({}_{2.5}\) becomes weaker as time goes on. The Sankey diagram supports the modeling of the spatiotemporal prediction by combining features, deep learning models, and interpolation models, as shown in Figure 1d. We set the prediction parameters for the models in (e). Here, we set the time lag and deep learning parameters. The interpolated prediction with the nearest neighbor is visualized in (f), where we see the predicted values over the global area. The observed ground truth data are visualized in (g), and the prediction errors are visualized in (h). The standard deviation of prediction over time is presented in (i). The LISA is shown in (j). The box plots represent the temporal predictions compared to the actual observed values in (k).