# Evaluating Machine Learning and Remote Sensing in Monitoring NO2 Emission of Power Plants
## 3 Materials and Methods
### Machine Learning Models  
#### 3.4.1 Long Short-Term Memory (LSTM)  
This section introduces an LSTM model used for this study. This was chosen as the first model based on one conclusion reached. EPA NO\({}_{2}\) target value affected subsequent values in the dataset due to the chronological nature of our data and the long-term effects of pollutants in the atmosphere. The conclusion that a current EPA NO\({}_{2}\) value for a day is affected by the previous day's NO\({}_{2}\) value, and an LSTM models' ability for insensitivity to gap duration in data with some missing days made this choice beneficial. LSTM traditionally is good with non-linear time-series data and can incorporate previous data by time steps, which fitted our data requirement. Three sets of input data were prepared:  
* The first set only contained two features (EPA NO\({}_{2}\), TROPOMI).
* The second input dataset included MERRA-2 (daily, weekly, monthly), TROPOMI NO\({}_{2}\), and date features (EPA\({}_{-}\)NO\({}_{2}\)10000, TROPOMI*1000, dayOfYear, dayOfWeek, dayOfMonth, MERRA-2 Wind, MERRA-2 Temp, MERRA-2 Precip, MERRA-2 Cloud Fraction).
* The third input set contained previously mentioned features and MCD19A2.  
The data was then reshaped to a 3-dimensional shape (adding a time axis) required by an LSTM model (number of samples, time steps, number of features). Reshaping must set a look back value specifying how many earlier days are used to create the time series. The input data has X predictors representing feature values at a given time (t) and Y (EPA NO\({}_{2}\)) representing the value of the next day (t + 1). The time step is measured in days and is set to one, whereas the \"look back\" value is one week (each time step looks at 7 values). The data was then split into training and testing, with a 66% training split. After scaling, reshaping, and splitting, an LSTM model architecture was built:  
The stacked LSTM architecture model comprised seven LSTM layers, each with (64, 128, 256, 128, 64, 4, 2) units and a dense layer with a single unit. Figure 6 visualizes the architecture of the stacked LSTM model.  
Figure 6: Stacked LSTM model architecture.  
The stacked LSTM model was constructed to be a deeper network topology. The ability for the model to extract and recombine higher-order features embedded in the data for performance improvement was one of the advantages of building a deeper network-especially for a stacked architecture. Each layer of the model passed a recurrent sequence to the next layer to account for the previous time step and to more accurately describe the complex pattern in the nonlinear data at every layer. The model used a dropout setting of 0.4 to reduce overfitting. An MSE loss function was utilized, along with an \"adam\" optimizer, since it is a typical optimizer in LSTM models and outperformed \"rmsprop\" in testing. A validation set was provided at training to track accuracy and loss to fine-tune the model. This validation set was not utilized for training but rather to measure the effectiveness of the hyperparameters provided during training.