# The Integration of Internet of Things and Machine Learning for Energy Prediction of Wind Turbines
## 3 ML for Wind Energy Prediction
### Dataset and Preprocessing
A correlation heatmap is a visual representation that presents the correlation between many variables in the form of a matrix, utilizing color codes. By utilizing the Python programming language and the seaborn library, it is possible to generate a helpful heatmap that displays the association between variables. The correlation coefficients for various variables are displayed in a correlation table. Figure 11: Box plots for numeric columns (time, year, month no needed to plot). \begin{table}
\begin{tabular}{l l} \hline \hline
**Columns** & **Removed Outliers** \\ \hline temperature\_2m & 5 \\ relativehumidity\_2m & 11 \\ dewpoint\_2m & 0 \\ windspeed\_10m & 318 \\ windspeed\_100m & 199 \\ winddirection\_10m & 0 \\ winddirection\_100m & 0 \\ winddistens\_10m & 337 \\ Power & Not included \\ Year & Not included \\ Month & Not included \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Removed outliers from columns. Conventional clustering and correlation analysis face difficulties when dealing with the vast amount and low density of valuable information in big data. To improve energy prediction, it is recommended to use big data-driven correlation analysis with clustering. Conducting correlation analysis among several measures such as temperature, dew point, relative humidity, wind direction, wind gusts, and wind speed will aid in the selection of forecast parameters. These strongly connected parameters contribute to the accurate prediction of the electricity produced by wind turbines. The heatmap in Figure 13 displays the relationships between every conceivable combination of values. It is a potent tool for detecting and visualizing patterns in data, as well as condensing large amounts of data. A Python program, utilizing the Seaborn module, may generate a heatmap that visually represents the association between variables [21]. Figure 12: Distribution of wind gusts (**a**) before and (**b**) after removal of outliers. Figure 13: Heatmap—correlation matrix. When preparing a dataset for ML models, preprocessing stages include data standardization and splitting. Consequently, all three regression models (linear regression, random forest regression, and lasso regression) follow this preprocessing phase. Standardization: This procedure guarantees that the dataset's features (or variables) have a mean of 0 and a standard deviation of 1 [4; 25]. This phase is essential as numerous ML algorithms exhibit enhanced performance when the data is normalized or standardized. The StandardScaler is a prevalent technique that subtracts the mean and normalizes data to unit variance. The standardization formula for each feature \(x\) is as follows:  
\[z=\left(x-\mu\right)\ /\ \sigma \tag{4}\]  
\(\mu\) is the mean value of the feature;  
\(z\) is the standardized feature;  
\(\sigma\) is the standard deviation of the feature. Splitting the data: This pertains to partitioning the dataset into training and testing (and occasionally validation) subsets [25]. The training set is utilized to develop the model, whereas the testing set is employed to assess its performance on novel data. The validation set is also employed to optimize the model without affecting the test set, particularly during hyperparameter optimization. This mitigates overfitting and guarantees the model generalizes effectively to novel inputs. The data are typically divided between 70% and 80% for training and 20% and 30% for assessment. The precise ratio is contingent upon the task at hand and the extent of the dataset. After completing the data purification procedure, the dataset (loc1) is now available and ready to be used for extracting meaningful insights.