# Application of Artificial Intelligence Models for Evapotranspiration Prediction along the Southern Coast of Turkey

Authors: Mohammed Majeed Hameed; Mohamed Khalid AlOmar; Siti Fatin Mohd Razali; Mohammed Abd Kareem Khalaf; Wajdi Jaber Baniya; Ahmad Sharafati; Mohammed Abdulhakim AlSaadi
Journal: Complexity
 (2021)

---

## Abstract

Reference evapotranspirationETo is one of the most significant factors in the hydrological cycle since it has a great influence on water resource planning and management, agriculture and irrigation management, and other processes in the hydrological sector. In this study, an efficient and local predictive model was established to forecast the monthly mean ETot over Turkey based on the data collected from 35 locations. For this purpose, twenty input combinations including hydrological and geographical parameters were introduced to three different approaches called multiple linear regression MLR, random forest RF, and extreme learning machine ELM. Moreover, in this study, large investigation was done, involving the establishment of 60 models and their assessment using ten statistical measures. The outcome of this study revealed that the ELM approach achieved high accurate estimation in accordance with the Penman–Monteith formula as compared to other models such as MLR and RF. Moreover, among the 10 statistical measures, the uncertainty at 95% U95 indicator showed an excellent ability to select the best and most efficient forecast model. The superiority of ELM in the prediction of mean monthly ETo over MLR and RF approaches is illustrated in the reduction of the U95 parameter to 49.02% and 34.07% for RF and MLR models, respectively. Furthermore, it is possible to develop a local predictive model with the help of computer to estimate the ETousing the simplest and cheapest meteorological and geographical variables with acceptable accuracy.


---

## Body

## 1. Introduction

### 1.1. Background

Global warming has become a great concern of researchers and world leaders. It is well known that the Earth surface temperature is increasing significantly during the last decades [1, 2]. Water storage, hydrological cycles, and, consequently, water availability are directly affected by global warming [3–5]. Thus, one of the most essential indicators of climate change is the referenced evapotranspiration ETo, which is considered as the most complicated element in the hydrological cycle [6–8].ETo mainly occurs due to two complicated processes. The first is when water evaporates from the surface of the soil, lakes, rivers, etc., and this process is called physical evaporation. The second process is the transpiration phenomenon in crops and plants, which is called biological transpiration [9, 10]. Undeniably, the evaporation process requires energy to convert water from the liquid phase to the vapor phase. Therefore, the main parameters that affect the ETo process is the sun radiation, wind speed and direction, air temperature, and humidity [9–11]. In conclusion, the EToalso represents the link between surface energy and carbon cycle [12, 13]. Based on the stated literature, a precise measurement and prediction of ETois essential for quantifying surface energy and water reserves worldwide [14–16]. Thus, providing accurate models for weather and climate change diagnosis is crucial [14, 17–21].


### 1.2. Literature Review

Due to the significant effect ofEToon climate change, Earth temperature, crops and plants, water management, and runoff quantity, many researchers have studied the EToprediction over the last decades [9, 22–24]. The Penman–Monteith P−M is the most widely used model, and it is considered as a physical ETomodel as it is an approximate linearized solution governing energy balance, thermodynamic state, vertical heat, and water-vapor diffusion [9, 23]. However, P−M requires many meteorological data to be applied, which can be considered as a drawback for this equation [25, 26]. In any case, there are many models applied to estimate the EToaround the world. Examples of such are the constant heat method by including heat pulse [27, 28] and the Shuttleworth–Wallace S-W method to estimate the transpiration from plants [29–31]. It is worth mentioning that the number of empirical equations for modeling the evaporation has exceeded 100 due to the importance of ETomeasurements and the variety of meteorological data around the world. Therefore, it is impossible to decisively compare these models [32–34].Recently, the development of artificial intelligence has received significant attention from communities in the hydrological and environmental sciences, including water treatment [35, 36], hydrology [37–41], water reservoir optimization [42–44], remote sensing applications [45, 46], etc. Consequently, due to the highly nonlinear characteristics associated with the ETodata, AI technology presented a suitable modeling approach to solve many issues with the empirical equations that has been used before [47]. Kumar et al. utilized the artificial neural network (ANN) in 2002 for predicting ETo, where different ANN architectures were implemented for evaporation simulation. The radial neural network yielded the best results for evaporation simulation, and it calculates the number of layers and neurons based on a trial and error process. [48]. Many researchers have followed his footprint in predicting ETo[49–52]. In addition, an adaptive network-based fuzzy inference system (ANFIS) has been used to predict ETo [53, 54]. The ANFIS and ANN techniques and empirical equations were used in the evapotranspiration field, and it was found that the ANFIS and ANN methods were much better than the empirical equations. [55, 56]. However, it is well established that ANN models easily get stuck in a local minimum, and, therefore, recent studies have employed new models adopting other AI techniques for ETomodeling [47]. Many approaches have been utilized for this purpose, including support vector machine SVM [53, 57, 58]. The SVM is well known to have a basic form, but one of the drawbacks of SVM is the unknown parameter [59]. Another approach in the field of simulation ETo data is genetic programming (GP). This approach consists of measurement programming capable of obtaining input data and producing a nonlinear interaction between data to determine the outputs [55, 60–62].In order to enhance the ability of AI models, many algorithms were associated with different AI methodologies, including the use of wavelet transform regression model [63], wavelet coupled with ANN [64, 65], and wavelet enhanced extreme learning machine [66]. Others utilize random forest (RF) algorithm to enhance the AI techniques. Due to its success over a variety of datasets, high precision estimation, a small range of user-defined parameters, the ability to estimate relative value of the variables, and its ability to preclude overfitting, the RF approach has become extremely popular in recent years [26, 34, 67, 68]. Recently, due to its higher efficiency and much quicker calculation speed, a newly proposed machine learning technology called the extreme learning machine (ELM) has confirmed it to be a promising ETo estimation tool [69]. First, Abdullah et al. (2015) used ELM to forecast ETo at three Iraqi stations and concluded that the ELM model is highly efficient and computerized at high generalization speeds [70, 71]. Ever since, the ELM for ETo predictions has been used by many studies in different climate environments [72–74]. To the best of the authors' knowledge, all models presented in the literature were established to simulate the evapotranspiration using a single model for each location or case study. Furthermore, some researchers employed modern techniques and used different case studies, but they could not produce a general model that could take into account more than one case study. In this study, an effort is being made to include a robust modeling methodology using a variety of locations along the southern coast of Turkey to create a comprehensive general model to forecast ETo. The other objective of the study is to predict mean monthly ETofrom limited data, which can be easily available.


### 1.3. Motivation of the Study

Due to the significance ofETo, there are numerous studies that have been conducted to estimate it, such as AI techniques and empirical models. In general, these approaches achieved a satisfactory success. However, creating one robust model for conducting local ETothroughout a specific country based on data collected from different sites is still considered as a challenging issue that needs to be addressed. Moreover, recognizing the statistical measures that are effectively used to assess the feasibility of a certain model are also very significant in the selection of the best predictive model. Thus, in this study, a broad investigation is performed using three different approaches, multiple linear regression (MLR), random forest (RF), and extreme learning machine (ELM), based on twenty combinations of meteorological and geographical indicators constituting 60 predictive models. Moreover, there were 10 efficient statistical measures employed to assess the accuracy of the performance for each model separately in accordance with the Penman–Monteith equation. Although the formula of Penman–Monteith is well-known in the prediction of ETo, it does pose some issues regarding the measurement of some factors, which may not be relevant to every site such as solar radiation and sensible heat flux into the soil. Therefore, a robust local model is established in this study, which can efficiently predict the local mean monthly ETo over the southern coast of Turkey using conveniently and inexpensively measurable parameters. Furthermore, the evaluation of the outcomes of the models will be further assessed and validated against the actual ETovalues, which are calculated by the Penman–Monteith equation.


## 1.1. Background

Global warming has become a great concern of researchers and world leaders. It is well known that the Earth surface temperature is increasing significantly during the last decades [1, 2]. Water storage, hydrological cycles, and, consequently, water availability are directly affected by global warming [3–5]. Thus, one of the most essential indicators of climate change is the referenced evapotranspiration ETo, which is considered as the most complicated element in the hydrological cycle [6–8].ETo mainly occurs due to two complicated processes. The first is when water evaporates from the surface of the soil, lakes, rivers, etc., and this process is called physical evaporation. The second process is the transpiration phenomenon in crops and plants, which is called biological transpiration [9, 10]. Undeniably, the evaporation process requires energy to convert water from the liquid phase to the vapor phase. Therefore, the main parameters that affect the ETo process is the sun radiation, wind speed and direction, air temperature, and humidity [9–11]. In conclusion, the EToalso represents the link between surface energy and carbon cycle [12, 13]. Based on the stated literature, a precise measurement and prediction of ETois essential for quantifying surface energy and water reserves worldwide [14–16]. Thus, providing accurate models for weather and climate change diagnosis is crucial [14, 17–21].


## 1.2. Literature Review

Due to the significant effect ofEToon climate change, Earth temperature, crops and plants, water management, and runoff quantity, many researchers have studied the EToprediction over the last decades [9, 22–24]. The Penman–Monteith P−M is the most widely used model, and it is considered as a physical ETomodel as it is an approximate linearized solution governing energy balance, thermodynamic state, vertical heat, and water-vapor diffusion [9, 23]. However, P−M requires many meteorological data to be applied, which can be considered as a drawback for this equation [25, 26]. In any case, there are many models applied to estimate the EToaround the world. Examples of such are the constant heat method by including heat pulse [27, 28] and the Shuttleworth–Wallace S-W method to estimate the transpiration from plants [29–31]. It is worth mentioning that the number of empirical equations for modeling the evaporation has exceeded 100 due to the importance of ETomeasurements and the variety of meteorological data around the world. Therefore, it is impossible to decisively compare these models [32–34].Recently, the development of artificial intelligence has received significant attention from communities in the hydrological and environmental sciences, including water treatment [35, 36], hydrology [37–41], water reservoir optimization [42–44], remote sensing applications [45, 46], etc. Consequently, due to the highly nonlinear characteristics associated with the ETodata, AI technology presented a suitable modeling approach to solve many issues with the empirical equations that has been used before [47]. Kumar et al. utilized the artificial neural network (ANN) in 2002 for predicting ETo, where different ANN architectures were implemented for evaporation simulation. The radial neural network yielded the best results for evaporation simulation, and it calculates the number of layers and neurons based on a trial and error process. [48]. Many researchers have followed his footprint in predicting ETo[49–52]. In addition, an adaptive network-based fuzzy inference system (ANFIS) has been used to predict ETo [53, 54]. The ANFIS and ANN techniques and empirical equations were used in the evapotranspiration field, and it was found that the ANFIS and ANN methods were much better than the empirical equations. [55, 56]. However, it is well established that ANN models easily get stuck in a local minimum, and, therefore, recent studies have employed new models adopting other AI techniques for ETomodeling [47]. Many approaches have been utilized for this purpose, including support vector machine SVM [53, 57, 58]. The SVM is well known to have a basic form, but one of the drawbacks of SVM is the unknown parameter [59]. Another approach in the field of simulation ETo data is genetic programming (GP). This approach consists of measurement programming capable of obtaining input data and producing a nonlinear interaction between data to determine the outputs [55, 60–62].In order to enhance the ability of AI models, many algorithms were associated with different AI methodologies, including the use of wavelet transform regression model [63], wavelet coupled with ANN [64, 65], and wavelet enhanced extreme learning machine [66]. Others utilize random forest (RF) algorithm to enhance the AI techniques. Due to its success over a variety of datasets, high precision estimation, a small range of user-defined parameters, the ability to estimate relative value of the variables, and its ability to preclude overfitting, the RF approach has become extremely popular in recent years [26, 34, 67, 68]. Recently, due to its higher efficiency and much quicker calculation speed, a newly proposed machine learning technology called the extreme learning machine (ELM) has confirmed it to be a promising ETo estimation tool [69]. First, Abdullah et al. (2015) used ELM to forecast ETo at three Iraqi stations and concluded that the ELM model is highly efficient and computerized at high generalization speeds [70, 71]. Ever since, the ELM for ETo predictions has been used by many studies in different climate environments [72–74]. To the best of the authors' knowledge, all models presented in the literature were established to simulate the evapotranspiration using a single model for each location or case study. Furthermore, some researchers employed modern techniques and used different case studies, but they could not produce a general model that could take into account more than one case study. In this study, an effort is being made to include a robust modeling methodology using a variety of locations along the southern coast of Turkey to create a comprehensive general model to forecast ETo. The other objective of the study is to predict mean monthly ETofrom limited data, which can be easily available.


## 1.3. Motivation of the Study

Due to the significance ofETo, there are numerous studies that have been conducted to estimate it, such as AI techniques and empirical models. In general, these approaches achieved a satisfactory success. However, creating one robust model for conducting local ETothroughout a specific country based on data collected from different sites is still considered as a challenging issue that needs to be addressed. Moreover, recognizing the statistical measures that are effectively used to assess the feasibility of a certain model are also very significant in the selection of the best predictive model. Thus, in this study, a broad investigation is performed using three different approaches, multiple linear regression (MLR), random forest (RF), and extreme learning machine (ELM), based on twenty combinations of meteorological and geographical indicators constituting 60 predictive models. Moreover, there were 10 efficient statistical measures employed to assess the accuracy of the performance for each model separately in accordance with the Penman–Monteith equation. Although the formula of Penman–Monteith is well-known in the prediction of ETo, it does pose some issues regarding the measurement of some factors, which may not be relevant to every site such as solar radiation and sensible heat flux into the soil. Therefore, a robust local model is established in this study, which can efficiently predict the local mean monthly ETo over the southern coast of Turkey using conveniently and inexpensively measurable parameters. Furthermore, the evaluation of the outcomes of the models will be further assessed and validated against the actual ETovalues, which are calculated by the Penman–Monteith equation.


## 2. Case Study and Data Collection

In this study, the data collected from 35 meteorological stations in Turkey are used. The geographical location of these stations covers large area located between a latitude from 36° to 38°. Figure1 shows the location of each meteorological station separately. It can be seen that the majority of these stations are located in the south of Turkey on the coasts of the Mediterranean Sea. The data collected from the general directorate of Turkish state meteorological includes several long-term monthly meteorological data such as temperature, humidity, wind speed, and reference evapotranspiration. Furthermore, the dataset comprises of the long-term mean monthly variables covering the period from 1975 to 2010. The highest temperature is recorded at the Mut station at 46.7°C, while the lowest temperature is recorded at Goksun station at −33.5°C. Figure 2 showed the long-term monthly mean (ETo), overall in Turkey. It can be observed that the highest value of (ETo) occurred in July followed by June, August, and May, consecutively.Figure 1
The distribution of meteorological stations in Turkey.Figure 2
Distribution of the mean monthlyETo over Turkey.It is worth mentioning that Turkey primarily has a complex climate due to its location and topography. The Mediterranean climate in southern Turkey is predominant with warm and dry summers and wet and moderate to cold winters. Continental weather is predominant in central Turkey with warm and dry summers and cold winters. The oceanic atmosphere in northern Turkey is seen through warm and rainy summers and cold and wet winters. This study ensures that the stations chosen to measureETo are spread nearly uniformly across southern Turkey in order to determine spatial differences in mean monthly ETo values and their time characteristics.Due to the lack ofETo data, the FAO embraced the PM equation as a standard methodology for calculating the reference ETo. The FAO56-PM can be used on hourly or daily scales to supply the data needed for machine learning approaches. The equation is suggested for hourly time measures as expressed in the following equation [7, 75]:(1)ETo=0.408ΔRN−G+γ900/T+273U2ea−edΔ+γ1+0.34U2,where ETo is the reference evapotranspiration in (mm/day), Δ represents the slope of the saturation vapor pressure function at air temperature T in kPa°⋅C−1, RN is the net solar radiation in MJ⋅m−2⋅day−1, G is the soil heat flux density in MJ⋅m−2⋅day−1, γ is the psychometric constant in kPa°⋅C−1, T is the mean air temperature in (°C), U2 is the average 24-hour wind speed at 2m height above the ground surface in m.s−1, ea is the saturation vapor pressure in kPa, and ed is the actual vapor pressure in kPa. Finally, it is important to mention that the dataset is subjected to the normalization process (between 0 and 1), for all input variables and their appropriate targets. This process is very important in order to boost the predictive model performance. Later, the data are simulated using three modeling approaches, namely, RF, ELM, and MLR.


## 3. Methodology

### 3.1. Random Forest

Random Forests (RF) is an algorithm that manages regression issues of high dimensions. This method is tree-based, where all trees have random variables in the selection, and the forest is established from several trees of regression and is clustered together [34, 76]. The tree is chosen as a random subset of variables that will be used to determine the result of the prediction. Whereas, in the random forest learning process, two significant parameters are identified: the first one is the number of trees ntree and the second one is the number of variables in each division mtry. After fitting single tree into the ensemble (bagging procedure), the final decision is made by averaging the output. The bias between the bagged trees is equal to that of the single tree, while the variance is reduced as the correlation between the trees is reduced [77].For regression-based RF formation, in order to get the tree predictorhX,⊖ numerical value, the process starts by growing trees on the bias of a random vector ⊖. For any given numerical estimator, the mean squared generalization error can be expressed as follows [78]:(2)EX,Y=Y−hX2.The RF predictor is generated by taking an average overj of a single tree. Here, in this respect, the following theorems line up:Theorem 1.
By expanding the number of trees in the forest, the error will then be expressed as follows:(3)EX,YYavjhX,⊖j2⊖EX.YY−E⊖hX,⊖2.This equation's right hand demonstrates the generalization error of the forest. Similarly, the average tree generalization error can be found from the following equation:(4)PE∗tree=E⊖EX,YY−hX,⊖2.Theorem 2.
If we assumeEY=EXhX,⊖ for every ⊖, then(5)PE∗forest≤p¯.PEtree,where p¯ represent the weight correlation [76].


### 3.2. Extreme Learning Machine (ELM)

The ELM, an advanced learning algorithm, is a machine learning modeling approach assembled with single-layer feedforward neural network (SLFN) [79]. ELM’s primary strength is that the weight of the input parameters is determined arbitrarily, while the output parameter weights are measured analytically using the Moore–Penrose approach (Huang et al., 2006). The SLFN function combines the hidden node additives and activation function, which can be represented mathematically as follows:(6)fx=∑i=1LβiGai,bi,x,wherefx represents the ELM model output function; x represents the input variable, ai and bi represent the learning parameters hidden nodes, and L stands for the number of hidden nodes. The βi governs the connecting weight to the i-hidden node between the output nodes. The output vector Gai,bi,x implies the output of the hidden node. The sigmoid activation solves the additional hidden node as follows:(7)Gai,bi,x=gai.x+bi,where ai and bi represent weight and basis values for each i-node in its hidden layer. In the algorithm of ELM, the weight of the input layer and bias values are randomly generated. Figure 3 showed the basic structure of the ELM.Figure 3
Fundamental structure of ELM.For the input and output variables, the arbitrary distinct sampleN is denoted as xi,yi. Based on the above findings, equation (6) can be rewritten as follows:(8)Hβ=T,where the Hussain matrix H is represented as(9)H=Ga1,b1+x1⋯gLaL,bL,x1⋮⋱⋮GaN,bN+xN⋯gLaL,bL,xLNxL.In addition,(10)β=β1T...βLTLxm,T=Y1T...YLTNxm.G represents the activation function and in this current study, log-sigmoid transfer function is used, which is expressed as(11)Gy=11+exp−y,where y=ax+b as indicated in equation (6).The ELM model with independentL sets, which can result in zero learning errors, can be trained in the form of SLFN [80]. In addition, even with fewer hidden neurons L than the number of separate samples N, the ELM will specify random parameters for the hidden nodes. Ultimately, the output weights can be calculated by means of pseudoinverse of H, which makes a limited error range ε>0. The hidden node parameters (weights and bias) are given random values during the training phase [81].


### 3.3. Multiple Linear Regression

In the form of the local climatic parameters of minimum and maximum temperature, relative humidity, wind speed and sun hours, theETodata can be formulated using multiple linear regression (MLR) models. Herein, the ETois calculated by MLR. The MLR can be employed to describe the relationship between dependent and independent variables as a multivariate statistical tool described by the following equation:(12)Y=αo+α1X1+α2X2+⋯αjX1,where the response variables are represented by Y, which is also the predicted mean monthly ETo.The independent variables and the predictors are represented by X1,X2,Xn, and the coefficients of regression are represented by αoα1,α2,…,αj, which can be acquired by the following equations [82]:(13)e=∑i=1nY−yi2,∂e∂αo=0,∂e∂α1=0,∂e∂α2=0,∂e∂αj=0,where the error of the estimated and real values of ETois represented by e and yi, respectively.


### 3.4. Model Evaluation

The selection process of the best predictive model is of great importance to achieve high accurate predictions. Therefore, in this study, ten statistical parameters have been used to assess the performances of each predictive approach [83, 84]. The quantitative parameters are as follows:(i)
Mean absolute errorMAE. It can be expressed by assuming the absolute errors divided by the number of total observations. This indicator is widely used in water resources and hydrological sectors to assess the predictive models because it provides significant information on how closely the simulated data points match with the actual ones. The mathematical expression of MAE is shown in the following equations [85, 86]:(14)e=EToi,m−EToi,p,where e is the forecasted error,(15)MAEmm/month=∑i=1nein.(ii)
Root mean square errorRMSE. It is a statistical parameter often used to compare the forecasting errors of several models. The lower RMSE value usually points out to better predictions. The RMSE can be derived using the following equation [87, 88]:(16)RMSEmm/month=1n∑i=1nei2.(iii)
Mean absolute relative errorMARE. This indicator can be expressed as an absolute relative computer error (difference between actual and forecasted points). When the MARE parameter is represented as a percentage, it is called the mean absolute relative error MARE, which is expressed by the following equation:(17)MARE=1n∑i=1neiEToi,m.(iv)
Root mean square relative errorRMSRE is expressed as(18)RMSRE=1n∑i=1neiEToi,m2.(v)
Relative root mean square errorRRMSE can be calculated by dividing the RMSE criteria over the mean of actual data points. This parameter is very vital in assessing the accuracy of a model. In accordance to [89], the model is considered excellent if RRMSE>10%, good if 10%<RRMSE<20%, fair if 20%<RRMSE<30%, and last, the model can be considered poor if RRMSE>30%. The mathematical expression of the RRMSE is depicted as follows:(19)RRMSRE%=RMSE1/n∑i=1nEToi,m.(vi)
Mean bias errorMBE. It discloses the tendency of a model and explains whether it overestimates the data or underestimates them and is expressed by the following equation:(20)MBEmm/month=1n∑inei.(vii)
Correlation of coefficientCC. The CC is a significant factor that can be utilized to efficiently discover the robustness of the relationship between predicted and simulated data points. The mathematical expression of CC is shown in the following equation [82, 90]:(21)CC=∑i=1nEToi,m−EToi,m¯EToi,p−EToi,p¯∑i=1nEToi,m−EToi,m¯2∑i=1nEToi,p−EToi,p¯2.(viii)
Maximum absolute relative errorerMAX is expressed as(22)erMAXeiEToi,m.(ix)
t-statistic t−stat. This statistical test is beneficial for validating and testing the broadband models [91]. As t−stat indicator approaches zero, the desired model would be achieved and it is expressed by the following equation:(23)t−stat=n−1MBE2MBE2−RMSE2,where n is the total number of observations, and EToi,mEToi,p are the ith actual and estimated mean monthly reference of evapotranspiration, respectively.(x)
Uncertainty at 95%U95. This quantitative criterion is very efficient in terms of selecting an efficient predictive model among several models. The U95indicator provides very useful information on the deviation of a certain model [91]. The U95 can be calculated by the following equation:(24)U95=1.96SD2+RMSE21/2, where SD represents the standard deviation of the difference between true and simulated data points. The value 1.96 is a coverage factor corresponding to 95% confidence level. Last, for visual evaluation, boxplots and scatter plots are presented in the Results and Discussion sections. The coefficient of determination R2 is calculated based on the following equation:(25)R2=1−∑i=1nei2∑i=1nEToi,p−EToi,p¯2.


### 3.5. Model Development

To accurately predict the monthlymeanETo, three different models were assigned called MLR, RF, and ELM approaches. In accordance with the nature of dataset collected from different locations and sites, and in addition to having different characters, it is a significant and difficult task to create a reliable predictive model. Therefore, the current work is carried out by establishing 20 different input groups for each predictive approach as shown in Table 1. The dataset is randomly divided into phases called training set and testing set. The training set is used for calibration process and model construction, while the testing set is used for examining the performance accuracy of each candidate model. Table 2 shows the statistical description of the variables used in this study. In addition, Table 3 gives information about the Pearson correlation coefficient between all variables and ETo used in the suggested case study. As shown in the table, the maximum temperature variable has the highest correlation with ETofor both training and testing datasets. The table shows that the longitude variable has the lowest correlation coefficient with an ETo of 0.09, while the maximum temperature is significantly correlated with ETowith an R of 0.841.Table 1
Statistical description of the predictors and evapotranspiration that used in this current study.
VariablesUnitMaxMinMeanStdSkewnessKurtosisLatitudeTraining—38.2136.0536.80.690.892.44Testing38.2136.0536.870.750.732.05Overall38.2136.0536.820.70.852.34LongitudeTraining—37.1129.0733.372.57−0.241.56Testing37.1129.0733.82.58−0.471.68Overall37.1129.0733.482.58−0.291.58ElevationTrainingm15523369.35489.370.972.33Testing15523426.25528.160.771.94Overall15523383.57499.320.922.22TmaxTrainingCO46.712.532.657.54−0.442.32Testing45.414.431.78.01−0.171.84Overall46.712.532.417.67−0.372.17TminTrainingCO20.6−33.51.4610.27−0.683.65Testing17−29.4−0.1111.03−0.673.15Overall20.6−33.51.0710.47−0.683.52TmeanTrainingCO30.2−3.816.787.81−0.312.35Testing28.9−0.515.738.25−0.162.01Overall30.2−3.816.527.93−0.272.25WsTrainingm/sec5.60.71.970.761.466.2Testing5.10.72.150.931.264.35Overall5.60.72.020.811.445.68RHmax%Training—88.453.873.337.62−0.392.52Testing90.155.875.177.19−0.442.8Overall90.153.873.797.55−0.412.59RHmin%Training—3807.765.671.436.47Testing2617.815.440.773.16Overall3807.775.611.285.75ETOTrainingmm/month15.981.56.472.790.272.59Testing14.81.456.52.980.262.34Overall15.981.456.482.840.272.52Table 2
Different input combinations.
CombinationsInput parametersC1EO=fLatitude,Longitude,Elevation,Tmax,Tmin,Tmean,Ws,RHmax,RHminC2EO=fLatitude,Tmax,Tmin,Tmean,Ws,RHmax,RHminC3EO=fLongitude,Tmax,Tmin,Tmean,Ws,RHmax,RHminC4EO=fElevation,Tmax,Tmin,Tmean,Ws,RHmax,RHminC5EO=fLatitude,Tmax,Ws,RHmaxC6EO=fLongitude,Tmax,Ws,RHmaxC7EO=fElevation,Tmax,Ws,RHmaxC8EO=fLatitude,Tmin,Ws,RHminC9EO=fLongitude,Tmin,Ws,RHminC10EO=fElevation,Tmin,Ws,RHminC11EO=fLatitude,Tmean,WsC12EO=fLongitude,Tmean,WsC13EO=fElevation,Tmean,WsC14EO=fTmax,Ws,RHmaxC15EO=fTmin,Ws,RHmaxC16EO=fTmean,Ws,RHmaxC7EO=fTmean,Ws,RHminC18EO=fTmax,Ws,LatitudeC19EO=fTmax,WsC20EO=fTmaxTable 3
Crosscorrelation matrix of the variables used in this study.
LatitudeLongitudeElevationTmaxTminTmeanWSRHmaxRHminETOTrainingLatitude1.000Longitude0.2061.000Elevation0.673−0.1121.000Tmax−0.1940.095−0.3011.000Tmin−0.468−0.021−0.5220.8521.000Tmean−0.3190.036−0.3930.9430.9541.000WS−0.1140.0040.0240.0270.1250.0891.000RHmax0.1150.157−0.012−0.279−0.341−0.339−0.0241.000RHmin0.000−0.0460.084−0.143−0.058−0.0680.1180.0791.000ETO−0.1010.052−0.0710.8150.6960.7780.459−0.208−0.0651.000TestingLatitude1.000Longitude0.1811.000Elevation0.717−0.0681.000Tmax−0.1680.166−0.3741.000Tmin−0.4500.064−0.6250.8771.000Tmean−0.2830.118−0.4930.9580.9511.000WS−0.3540.013−0.1690.0240.1760.1131.000RHmax0.1940.2170.083−0.175−0.268−0.237−0.0051.000RHmin0.157−0.0420.164−0.279−0.210−0.2330.0930.2041.000ETO−0.1660.091−0.2060.8410.7380.8200.419−0.144−0.2081.000The process of selecting a more accurate model is relatively tough with ten statistical matrices and three different approaches including several input combinations. Thus, the assessment process is carried out based on two stages. First, during the training set, from each approach, the best three different models (in total nine models) are being selected. In the second step, it is crucial to monitor the performance of those models, which are selected throughout the training phase, during the testing phase and to select the three most efficient models for each approach. Finally, this process would provide much information about each adopted approach, apart from providing a clear and realistic impression of the performance of each predictive model separately.Furthermore, a robust and effective performance measure is used to assess the capability of each model. Among these statistical metrics, the uncertainty at 95%U95 has been used to assess the performance accuracy for each model and subsequently recognize the best predictive model. Figure 4 describes the prediction process of ETo along the southern coast of Turkey. Last, it is imperative to emphasize that all input variables and their corresponding targets are normalized between 0 and 1. This process is very important to enhance the effectiveness of the predictive models [92, 93]. All models are developed using MATLAB 2017a.Figure 4
Prediction process of the monthly mean evapotranspiration using several AI approaches.


## 3.1. Random Forest

Random Forests (RF) is an algorithm that manages regression issues of high dimensions. This method is tree-based, where all trees have random variables in the selection, and the forest is established from several trees of regression and is clustered together [34, 76]. The tree is chosen as a random subset of variables that will be used to determine the result of the prediction. Whereas, in the random forest learning process, two significant parameters are identified: the first one is the number of trees ntree and the second one is the number of variables in each division mtry. After fitting single tree into the ensemble (bagging procedure), the final decision is made by averaging the output. The bias between the bagged trees is equal to that of the single tree, while the variance is reduced as the correlation between the trees is reduced [77].For regression-based RF formation, in order to get the tree predictorhX,⊖ numerical value, the process starts by growing trees on the bias of a random vector ⊖. For any given numerical estimator, the mean squared generalization error can be expressed as follows [78]:(2)EX,Y=Y−hX2.The RF predictor is generated by taking an average overj of a single tree. Here, in this respect, the following theorems line up:Theorem 1.
By expanding the number of trees in the forest, the error will then be expressed as follows:(3)EX,YYavjhX,⊖j2⊖EX.YY−E⊖hX,⊖2.This equation's right hand demonstrates the generalization error of the forest. Similarly, the average tree generalization error can be found from the following equation:(4)PE∗tree=E⊖EX,YY−hX,⊖2.Theorem 2.
If we assumeEY=EXhX,⊖ for every ⊖, then(5)PE∗forest≤p¯.PEtree,where p¯ represent the weight correlation [76].


## 3.2. Extreme Learning Machine (ELM)

The ELM, an advanced learning algorithm, is a machine learning modeling approach assembled with single-layer feedforward neural network (SLFN) [79]. ELM’s primary strength is that the weight of the input parameters is determined arbitrarily, while the output parameter weights are measured analytically using the Moore–Penrose approach (Huang et al., 2006). The SLFN function combines the hidden node additives and activation function, which can be represented mathematically as follows:(6)fx=∑i=1LβiGai,bi,x,wherefx represents the ELM model output function; x represents the input variable, ai and bi represent the learning parameters hidden nodes, and L stands for the number of hidden nodes. The βi governs the connecting weight to the i-hidden node between the output nodes. The output vector Gai,bi,x implies the output of the hidden node. The sigmoid activation solves the additional hidden node as follows:(7)Gai,bi,x=gai.x+bi,where ai and bi represent weight and basis values for each i-node in its hidden layer. In the algorithm of ELM, the weight of the input layer and bias values are randomly generated. Figure 3 showed the basic structure of the ELM.Figure 3
Fundamental structure of ELM.For the input and output variables, the arbitrary distinct sampleN is denoted as xi,yi. Based on the above findings, equation (6) can be rewritten as follows:(8)Hβ=T,where the Hussain matrix H is represented as(9)H=Ga1,b1+x1⋯gLaL,bL,x1⋮⋱⋮GaN,bN+xN⋯gLaL,bL,xLNxL.In addition,(10)β=β1T...βLTLxm,T=Y1T...YLTNxm.G represents the activation function and in this current study, log-sigmoid transfer function is used, which is expressed as(11)Gy=11+exp−y,where y=ax+b as indicated in equation (6).The ELM model with independentL sets, which can result in zero learning errors, can be trained in the form of SLFN [80]. In addition, even with fewer hidden neurons L than the number of separate samples N, the ELM will specify random parameters for the hidden nodes. Ultimately, the output weights can be calculated by means of pseudoinverse of H, which makes a limited error range ε>0. The hidden node parameters (weights and bias) are given random values during the training phase [81].


## 3.3. Multiple Linear Regression

In the form of the local climatic parameters of minimum and maximum temperature, relative humidity, wind speed and sun hours, theETodata can be formulated using multiple linear regression (MLR) models. Herein, the ETois calculated by MLR. The MLR can be employed to describe the relationship between dependent and independent variables as a multivariate statistical tool described by the following equation:(12)Y=αo+α1X1+α2X2+⋯αjX1,where the response variables are represented by Y, which is also the predicted mean monthly ETo.The independent variables and the predictors are represented by X1,X2,Xn, and the coefficients of regression are represented by αoα1,α2,…,αj, which can be acquired by the following equations [82]:(13)e=∑i=1nY−yi2,∂e∂αo=0,∂e∂α1=0,∂e∂α2=0,∂e∂αj=0,where the error of the estimated and real values of ETois represented by e and yi, respectively.


## 3.4. Model Evaluation

The selection process of the best predictive model is of great importance to achieve high accurate predictions. Therefore, in this study, ten statistical parameters have been used to assess the performances of each predictive approach [83, 84]. The quantitative parameters are as follows:(i)
Mean absolute errorMAE. It can be expressed by assuming the absolute errors divided by the number of total observations. This indicator is widely used in water resources and hydrological sectors to assess the predictive models because it provides significant information on how closely the simulated data points match with the actual ones. The mathematical expression of MAE is shown in the following equations [85, 86]:(14)e=EToi,m−EToi,p,where e is the forecasted error,(15)MAEmm/month=∑i=1nein.(ii)
Root mean square errorRMSE. It is a statistical parameter often used to compare the forecasting errors of several models. The lower RMSE value usually points out to better predictions. The RMSE can be derived using the following equation [87, 88]:(16)RMSEmm/month=1n∑i=1nei2.(iii)
Mean absolute relative errorMARE. This indicator can be expressed as an absolute relative computer error (difference between actual and forecasted points). When the MARE parameter is represented as a percentage, it is called the mean absolute relative error MARE, which is expressed by the following equation:(17)MARE=1n∑i=1neiEToi,m.(iv)
Root mean square relative errorRMSRE is expressed as(18)RMSRE=1n∑i=1neiEToi,m2.(v)
Relative root mean square errorRRMSE can be calculated by dividing the RMSE criteria over the mean of actual data points. This parameter is very vital in assessing the accuracy of a model. In accordance to [89], the model is considered excellent if RRMSE>10%, good if 10%<RRMSE<20%, fair if 20%<RRMSE<30%, and last, the model can be considered poor if RRMSE>30%. The mathematical expression of the RRMSE is depicted as follows:(19)RRMSRE%=RMSE1/n∑i=1nEToi,m.(vi)
Mean bias errorMBE. It discloses the tendency of a model and explains whether it overestimates the data or underestimates them and is expressed by the following equation:(20)MBEmm/month=1n∑inei.(vii)
Correlation of coefficientCC. The CC is a significant factor that can be utilized to efficiently discover the robustness of the relationship between predicted and simulated data points. The mathematical expression of CC is shown in the following equation [82, 90]:(21)CC=∑i=1nEToi,m−EToi,m¯EToi,p−EToi,p¯∑i=1nEToi,m−EToi,m¯2∑i=1nEToi,p−EToi,p¯2.(viii)
Maximum absolute relative errorerMAX is expressed as(22)erMAXeiEToi,m.(ix)
t-statistic t−stat. This statistical test is beneficial for validating and testing the broadband models [91]. As t−stat indicator approaches zero, the desired model would be achieved and it is expressed by the following equation:(23)t−stat=n−1MBE2MBE2−RMSE2,where n is the total number of observations, and EToi,mEToi,p are the ith actual and estimated mean monthly reference of evapotranspiration, respectively.(x)
Uncertainty at 95%U95. This quantitative criterion is very efficient in terms of selecting an efficient predictive model among several models. The U95indicator provides very useful information on the deviation of a certain model [91]. The U95 can be calculated by the following equation:(24)U95=1.96SD2+RMSE21/2, where SD represents the standard deviation of the difference between true and simulated data points. The value 1.96 is a coverage factor corresponding to 95% confidence level. Last, for visual evaluation, boxplots and scatter plots are presented in the Results and Discussion sections. The coefficient of determination R2 is calculated based on the following equation:(25)R2=1−∑i=1nei2∑i=1nEToi,p−EToi,p¯2.


## 3.5. Model Development

To accurately predict the monthlymeanETo, three different models were assigned called MLR, RF, and ELM approaches. In accordance with the nature of dataset collected from different locations and sites, and in addition to having different characters, it is a significant and difficult task to create a reliable predictive model. Therefore, the current work is carried out by establishing 20 different input groups for each predictive approach as shown in Table 1. The dataset is randomly divided into phases called training set and testing set. The training set is used for calibration process and model construction, while the testing set is used for examining the performance accuracy of each candidate model. Table 2 shows the statistical description of the variables used in this study. In addition, Table 3 gives information about the Pearson correlation coefficient between all variables and ETo used in the suggested case study. As shown in the table, the maximum temperature variable has the highest correlation with ETofor both training and testing datasets. The table shows that the longitude variable has the lowest correlation coefficient with an ETo of 0.09, while the maximum temperature is significantly correlated with ETowith an R of 0.841.Table 1
Statistical description of the predictors and evapotranspiration that used in this current study.
VariablesUnitMaxMinMeanStdSkewnessKurtosisLatitudeTraining—38.2136.0536.80.690.892.44Testing38.2136.0536.870.750.732.05Overall38.2136.0536.820.70.852.34LongitudeTraining—37.1129.0733.372.57−0.241.56Testing37.1129.0733.82.58−0.471.68Overall37.1129.0733.482.58−0.291.58ElevationTrainingm15523369.35489.370.972.33Testing15523426.25528.160.771.94Overall15523383.57499.320.922.22TmaxTrainingCO46.712.532.657.54−0.442.32Testing45.414.431.78.01−0.171.84Overall46.712.532.417.67−0.372.17TminTrainingCO20.6−33.51.4610.27−0.683.65Testing17−29.4−0.1111.03−0.673.15Overall20.6−33.51.0710.47−0.683.52TmeanTrainingCO30.2−3.816.787.81−0.312.35Testing28.9−0.515.738.25−0.162.01Overall30.2−3.816.527.93−0.272.25WsTrainingm/sec5.60.71.970.761.466.2Testing5.10.72.150.931.264.35Overall5.60.72.020.811.445.68RHmax%Training—88.453.873.337.62−0.392.52Testing90.155.875.177.19−0.442.8Overall90.153.873.797.55−0.412.59RHmin%Training—3807.765.671.436.47Testing2617.815.440.773.16Overall3807.775.611.285.75ETOTrainingmm/month15.981.56.472.790.272.59Testing14.81.456.52.980.262.34Overall15.981.456.482.840.272.52Table 2
Different input combinations.
CombinationsInput parametersC1EO=fLatitude,Longitude,Elevation,Tmax,Tmin,Tmean,Ws,RHmax,RHminC2EO=fLatitude,Tmax,Tmin,Tmean,Ws,RHmax,RHminC3EO=fLongitude,Tmax,Tmin,Tmean,Ws,RHmax,RHminC4EO=fElevation,Tmax,Tmin,Tmean,Ws,RHmax,RHminC5EO=fLatitude,Tmax,Ws,RHmaxC6EO=fLongitude,Tmax,Ws,RHmaxC7EO=fElevation,Tmax,Ws,RHmaxC8EO=fLatitude,Tmin,Ws,RHminC9EO=fLongitude,Tmin,Ws,RHminC10EO=fElevation,Tmin,Ws,RHminC11EO=fLatitude,Tmean,WsC12EO=fLongitude,Tmean,WsC13EO=fElevation,Tmean,WsC14EO=fTmax,Ws,RHmaxC15EO=fTmin,Ws,RHmaxC16EO=fTmean,Ws,RHmaxC7EO=fTmean,Ws,RHminC18EO=fTmax,Ws,LatitudeC19EO=fTmax,WsC20EO=fTmaxTable 3
Crosscorrelation matrix of the variables used in this study.
LatitudeLongitudeElevationTmaxTminTmeanWSRHmaxRHminETOTrainingLatitude1.000Longitude0.2061.000Elevation0.673−0.1121.000Tmax−0.1940.095−0.3011.000Tmin−0.468−0.021−0.5220.8521.000Tmean−0.3190.036−0.3930.9430.9541.000WS−0.1140.0040.0240.0270.1250.0891.000RHmax0.1150.157−0.012−0.279−0.341−0.339−0.0241.000RHmin0.000−0.0460.084−0.143−0.058−0.0680.1180.0791.000ETO−0.1010.052−0.0710.8150.6960.7780.459−0.208−0.0651.000TestingLatitude1.000Longitude0.1811.000Elevation0.717−0.0681.000Tmax−0.1680.166−0.3741.000Tmin−0.4500.064−0.6250.8771.000Tmean−0.2830.118−0.4930.9580.9511.000WS−0.3540.013−0.1690.0240.1760.1131.000RHmax0.1940.2170.083−0.175−0.268−0.237−0.0051.000RHmin0.157−0.0420.164−0.279−0.210−0.2330.0930.2041.000ETO−0.1660.091−0.2060.8410.7380.8200.419−0.144−0.2081.000The process of selecting a more accurate model is relatively tough with ten statistical matrices and three different approaches including several input combinations. Thus, the assessment process is carried out based on two stages. First, during the training set, from each approach, the best three different models (in total nine models) are being selected. In the second step, it is crucial to monitor the performance of those models, which are selected throughout the training phase, during the testing phase and to select the three most efficient models for each approach. Finally, this process would provide much information about each adopted approach, apart from providing a clear and realistic impression of the performance of each predictive model separately.Furthermore, a robust and effective performance measure is used to assess the capability of each model. Among these statistical metrics, the uncertainty at 95%U95 has been used to assess the performance accuracy for each model and subsequently recognize the best predictive model. Figure 4 describes the prediction process of ETo along the southern coast of Turkey. Last, it is imperative to emphasize that all input variables and their corresponding targets are normalized between 0 and 1. This process is very important to enhance the effectiveness of the predictive models [92, 93]. All models are developed using MATLAB 2017a.Figure 4
Prediction process of the monthly mean evapotranspiration using several AI approaches.


## 4. Results and Discussion

This section of the study is dedicated to illustrating the forecast results obtained for mean monthlyEToover Turkey via three different predictive models, namely, MLR, RF, and ELM. Twenty scenarios have been presented including different input variables and are introduced to the mentioned models. 60 predictive models are assessed and validated against the Penman–Monteith equation using ten efficient statistical indicators and graphical presentations. A qualified model is one that meets the requirements of most of the mentioned statistical parameters. The dataset collected from 35 stations is divided randomly into two sets: the training phase (75%) is used to calibrate the models, and the rest of the data is used for validation purposes.In terms of quantitative assessment, Table4 exhibits further information about the performances of the proposed 60 different models based on different input variables through the training phase. The three predictive modeling approaches have achieved different accuracies in accordance with statistical measures. Moreover, it is difficult to rank the models in accordance with the achieved accuracies, but the RF approach relatively showed the best accuracy predictions. However, for conducting a fair comparative analysis for each adopted approach, from each modeling technique, the best three models with different input combinations have been selected for performing further and fair comparisons.Table 4
Performance skill description of each predictive model according to different input combinations: training set.
ModelMAE (mm/month)RMSE (mm/month)MARERMSRERRMSE (%)CCerMAXMBE (mm/month)t_statU95(%)MLR-C10.8010.9380.1580.2214.4900.9421.6926.4717.9096.077RF-C10.5610.6880.1020.13410.6280.9721.5736.46817.8211.758ELM-C10.7220.8510.1410.19313.1540.9521.7766.4717.8754.126MLR-C20.8391.0110.1640.23215.6300.9321.8386.4717.9418.227RF-C20.5340.6710.0950.12410.3730.9731.5296.46917.8161.596ELM-C20.7720.930.1440.20214.3800.9431.6216.4717.9065.893MLR-C30.841.020.1640.2415.7660.9311.8156.4717.9448.516RF-C30.5410.690.0970.12710.6630.9711.56.47117.8221.781ELM-C30.8220.990.1590.22215.3030.9351.7266.4717.9317.557MLR-C40.8010.9380.1580.21914.5030.9421.6986.4717.9096.1RF-C40.5250.660.0950.12610.1970.9741.5316.48117.8131.49ELM-C40.7310.860.1350.1813.2930.9511.5916.4717.8794.305MLR-C50.8421.0140.1650.23515.6780.9311.8296.4717.9428.325RF-C50.5650.7050.1030.13410.8900.9711.6896.48417.8261.937ELM-C50.7610.9230.1360.17514.2630.9441.5326.4717.9035.704MLR-C60.8761.0530.1730.25216.2700.9261.7996.4717.9599.657RF-C60.5560.6970.1010.1310.7790.9711.4896.46517.8241.861ELM-C60.8160.9680.1560.21314.9560.9381.7626.4717.9226.896MLR-C70.80.9420.1580.2214.5590.9411.7056.4717.9116.194RF-C70.5240.6540.0950.12410.1120.9751.5316.47217.8111.441ELM-C70.7880.9230.1560.21414.2700.9441.6356.4717.9035.716MLR-C81.2481.5080.2540.35223.3100.8412.3966.4718.22240.689RF-C80.8311.0060.1590.20915.5420.9381.646.46417.9388.043ELM-C81.0881.3460.2050.27820.8090.8762.1136.4718.11725.846MLR-C91.4361.6870.2980.41226.0690.7962.3526.4718.35563.653RF-C90.8431.020.1660.22515.7690.9361.896.4717.9458.52ELM-C91.1331.3820.2160.29421.3570.8682.1056.4718.13828.67MLR-C101.2161.4610.2450.33722.5860.8522.256.4718.1935.863RF-C100.8010.9750.1570.21215.0760.9431.8986.48217.9247.118ELM-C101.0861.3410.2050.27220.7240.8771.956.4718.11325.431MLR-C111.0711.260.20.25219.4730.8921.8526.4718.06619.818RF-C110.8361.0130.1590.2115.6590.9431.8296.46317.9428.284ELM-C111.0041.2060.1770.21918.6450.9011.6886.4718.03616.653MLR-C121.1691.3670.2230.2921.1300.8711.9426.4718.12927.486RF-C120.8371.0050.1590.2115.5390.941.7936.46117.9388.033ELM-C120.9761.1890.1760.23218.3800.9041.9126.4718.02715.735MLR-C131.0131.2080.180.22518.6760.9011.8076.4718.03716.768RF-C130.7830.9340.1510.20214.4270.9521.8496.47817.9075.973ELM-C130.8981.0730.1610.20316.5910.9231.7426.4717.96910.441MLR-C140.8751.0570.1720.24916.3300.9251.8226.4717.9619.801RF-C140.6690.8370.1220.1612.9400.9611.7266.44917.8713.862ELM-C140.750.9210.1310.16514.2350.9441.4576.4717.9025.66MLR-C151.4511.7060.2970.40526.3620.7912.2716.4718.3766.586RF-C150.9761.1890.1920.25818.3830.9131.9866.46718.02815.74ELM-C151.2321.5020.2360.32423.2220.8422.3016.4718.21840.075MLR-C161.1621.360.220.28721.0110.8731.9076.4718.12526.868RF-C160.8451.0450.1610.21716.1500.9351.8246.46917.9569.381ELM-C160.9991.2010.1810.2318.5630.9021.8236.4718.03316.366MLR-C171.161.3580.2260.320.9880.8731.9726.4718.12426.749RF-C170.8671.0520.1650.21816.2540.9361.86.46317.9599.622ELM-C171.0381.2470.1870.23919.2780.8941.9026.4718.05919.034MLR-C181.0711.260.20.25219.4730.8921.8526.4718.06619.818RF-C180.8240.9890.1560.20515.2810.9421.7416.47117.9317.516ELM-C180.9431.1410.1670.2117.6370.9121.736.4718.00213.341MLR-C191.1691.3690.2230.29121.1550.8711.9536.4718.1327.616RF-C190.8271.0220.1490.19715.7940.9331.7956.46817.9458.577ELM-C191.051.2740.1860.23619.6840.8891.8666.4718.07420.69MLR-C201.3941.7510.2730.3827.0650.7782.6516.4718.40773.96RF-C201.0921.4250.1990.27922.0280.862.2976.44518.1732.439ELM-C201.3291.7090.250.34726.4160.792.4096.4718.37367.125Thus, among the 60 predictive models, only the best nine models are selected for optimally carrying out the efficient quantitative analysis. Reducing the number of models has many advantages. For instance, it ensures conducting a powerful and excellent compilation and, thereby, optimally choosing the best statistical matrices. Table5 shows the performance accuracy of three different approaches based on several input variables.Table 5
Selection of the best three models for each predictive approach through the calibration process.
ModelMAE (mm/month)RMSE (mm/month)MARERMSRERRMSE (%)CCerMAXMBE (mm/month)t_statU95(%)RankMLR-C10.8010.9380.1580.2214.4900.9421.6926.4717.9096.0771MLR-C40.8010.9380.1580.21914.5030.9421.6986.4717.9096.12MLR-C70.80.9420.1580.2214.5590.9411.7056.4717.9116.1943RF-C70.5240.6540.0950.12410.1120.9751.5316.47217.8111.4411RF-C40.5250.660.0950.12610.1970.9741.5316.48117.8131.492RF-C20.5340.6710.0950.12410.3730.9731.5296.46917.8161.5963ELM-C10.7220.8510.1410.19313.1540.9521.7766.4717.8754.1261ELM-C40.7310.860.1350.1813.2930.9511.5916.4717.8794.3052ELM-C140.750.9210.1310.16514.2350.9441.4576.4717.9025.663In general, remarkably, it can be noted from Table5 that the most frequent combination is C1, C4, and C7, consecutively. This means that the component includes all useful parameters that have effective impact on mean monthlyETo. At a glance, it can be said that the RF models provided more accurate estimations of mean monthly ETo than MLR and ELM approaches. Here, it is essential to mention that the uncertainty at 95% U95 indicator is the most efficient factor, which plays a major role in the evaluation process of the best model. Furthermore, the RF-C7 produced the lowest value (1.441) of U95 as compared with other predictive models. Besides, the model mentioned also presented high performance based on the rest of the statistical measures. Nevertheless, the accuracy of MLR was unsatisfactory and recorded a high uncertainty with U95of 6.077. This is due to the fact that the nonlinear relationship between predictors and their corresponding targets was not considered. Finally, the performances of ELM modeling approaches were satisfactory according to the U95indicator. With respect to the ELM approach, the best model was ELM-C1, which recorded 4.126 ofU95.Essentially, it should be taken into consideration that promising estimations were obtained during the training phase, which is primarily employed to effectively calibrate the models based on the known input variables and targets. However, the testing step is vital in assessing the performance of a model since it examines the model’s accuracy based on unseen target values. This advantage does not exist in the training set. Therefore, the reliable model should have a stable and balanced performance in both training and testing phases.After demonstrating the performance of the models during the calibration (training) phase, it is very important to see the accuracy of the adopted models during the testing phase. Table6 demonstrated the performance skill of each predictive model using different input parameters.Table 6
Performance skill description of each predictive model according to different input combinations: testing set.
ModelMAE (mm/month)RMSE (mm/month)MARERMSRERRMSE (%)CCerMAXMBE (mm/month)t_statU95 (%)MLR-C11.0781.2840.1720.21418.3920.9451.4496.25410.4215.151RF-C11.2811.5610.1870.21322.3590.9381.5565.99210.56329.955ELM-C10.9461.1550.1490.19116.5440.9561.6346.33310.3729.989MLR-C21.1171.3580.1730.21619.4580.9361.5456.24210.44819.537RF-C21.2291.5130.1780.20621.6730.9381.5036.03210.53526.723ELM-C20.9621.2160.1460.1817.4170.951.4226.32210.39212.574MLR-C31.1671.4070.1810.2220.1590.9361.5016.15810.47521.306RF-C31.2531.5410.1820.20922.0800.9351.4926.01810.5528.899ELM-C31.1111.3570.1680.20319.4440.9461.4566.13110.45817.33MLR-C41.0741.2820.1720.21418.3710.9451.4566.26610.41915.248RF-C41.2561.5230.1840.2121.8200.9391.5166.01310.54227.104ELM-C40.9991.2480.1470.17717.8840.9551.4676.22810.40912.848MLR-C51.1181.3480.1780.22119.3120.9341.566.28310.44119.589RF-C51.1441.4230.1680.19820.3910.9461.5266.16810.48122.741ELM-C51.0521.2690.1630.1918.1860.9521.4476.19710.41913.441MLR-C61.1991.4290.1950.24620.4750.9261.5516.23210.47724.555RF-C61.1341.4220.1640.19520.3710.9471.6776.15510.48222.346ELM-C61.0661.2820.1740.21818.3670.951.4966.19710.42314.126MLR-C71.0711.2810.1680.20418.3480.9451.4616.27410.41715.282RF-C71.1031.3810.160.19419.7850.9511.6496.16510.46419.593ELM-C71.0471.2530.1640.19917.9470.9561.4456.18510.41412.401MLR-C81.5241.8030.2760.36625.8370.8511.8716.34810.63673.725RF-C81.3891.6840.2170.27224.1320.9011.7846.25510.58952.417ELM-C81.3161.6330.2180.27323.3960.8911.5546.25610.56445.735MLR-C91.7512.0690.3150.41929.6440.7962.2676.29610.798129.564RF-C91.4681.7810.2290.27925.5120.8861.7276.20710.64665.221ELM-C91.3681.7310.2120.2824.8040.8722.1886.30210.60660.569MLR-C101.4621.7460.2640.36725.0150.8671.8116.28310.61662.258RF-C101.4141.6980.2210.27424.3210.9062.0156.24110.59853.811ELM-C101.2371.560.1980.24922.3560.9031.5936.29610.52638.316MLR-C111.3421.660.2130.26923.7850.9011.666.08310.643.783RF-C111.4671.8360.2090.25126.2980.9111.7675.95810.71964.059ELM-C111.2461.5830.1790.21322.6770.9291.3495.97610.57631.619MLR-C121.5051.8010.2480.30725.8060.8761.7886.05710.68162.81RF-C121.4081.7630.20.23425.2540.9171.5245.97610.67353.541ELM-C121.3021.6050.2010.2622.9970.9271.4765.94710.59132.973MLR-C131.2641.5890.1960.24622.7650.9181.5776.04410.5734.417RF-C131.4231.7830.2020.23925.5470.9191.5465.94410.6955.172ELM-C131.2771.5270.190.22721.8800.9491.3025.87810.56123.495MLR-C141.1971.4290.1940.24420.4700.9251.5696.24810.47624.857RF-C141.2471.550.1840.2222.2100.941.6226.14610.53933.389ELM-C141.0151.2590.1480.17618.0380.9521.3976.20910.41413.109MLR-C151.7342.0790.3080.40929.7920.7892.0796.33910.795134.22RF-C151.5991.9240.2660.33627.5650.8582.0486.26410.71693.894ELM-C151.4191.760.2290.29825.2200.8581.9796.38210.6167.441MLR-C161.4871.8050.2440.30525.8540.8781.7766.03410.68762.369RF-C161.5181.8880.2270.2827.0530.8912.0485.99410.74574.975ELM-C161.3511.6330.2070.25123.3940.9231.525.93210.60835.411MLR-C171.5341.8250.2620.33826.1490.8791.6635.98510.70863.591RF-C171.5391.8960.2290.27427.1660.9011.8585.96210.75674.846ELM-C171.3631.6720.2050.24123.9540.9161.445.93610.62840.003MLR-C181.3421.660.2130.26923.7850.9011.666.08310.643.783RF-C181.4071.7750.2030.24525.4260.9151.6565.95710.68354.491ELM-C181.2541.5770.1850.22722.5900.9291.5355.97510.57331.011MLR-C191.5051.8050.2480.30925.8670.8771.7766.04710.68563.046RF-C191.3691.7360.1950.2324.8750.9061.495.9910.65550.356ELM-C191.2651.5830.1860.22122.6740.9231.456.02910.56933.28MLR-C201.8772.3290.2950.36133.3660.822.1975.56711.228154.758RF-C201.8592.3980.2640.31634.3630.8132.0895.49911.333171.226ELM-C201.82.2750.2610.31732.5990.8322.1095.61311.156142.188Bold numbers refer to the best results.It is also important to carefully follow-up the performance of nine efficient models that were chosen in the training set (MLR-C1, MLR-C4, MLR-C7, RF-C7, RF-C4, RF-C2, ELM-C1, ELM-C4, and ELM-C14). Besides the mentioned models, there were additional three models (RF-C5, RF-C6, and ELM-C2) that have effectively provided satisfactory estimations. The heat-map diagram as presented in Figure5 provided significant information about the best modeling performance based on ten statistical parameters.Figure 5
Heat-map diagram illustrating the best three models for each predictive approach through the calibration process.Although RF models generated acceptable precision during the training set, it exhibited the worst accuracies as compared to ELM and MLR techniques in the testing set. Moreover, these models produced high uncertainty, and the values ofU95 are 19.59, 27.1, 22.74, 22.35, and 26.72% for RF-C7, RF-C4, RF-C5, RF-C6, and RF-C2, respectively. Additionally, the other statistical parameters such as RMSE and RRMSE also gave further information about the weaknesses of RF models. It is undeniable that the RF approach suffers from overfitting issue. On the other hand, the MLR models showed much better performance capacity than RF models. Finally, the ELM models achieved high precision in the prediction of mean monthly ETo in accordance with the P-M equation. Moreover, the ELM-C1 is considered the best predictive model and recorded the highest CC (0.957), lowest RMSE (1.155), MAE (0.946 mm/month), t−stat(10.37), RRMSE (16.54%), and U95(9.989%), respectively. Moreover, among ten statistical parameters, the most efficient parameters, which can easily recognize the best predictive model areU95, RMSE, and RRMSE. The supremacy of ELM approaches was evaluated in accordance with its ability of reducing the most significant statistical measures (U95, RMSE, and RRMSE) throughout the testing phase. The results obtained as shown in Figure 6 illustrated the superiority of ELM-C1 over other predictive models in reducing the value of the mentioned three statistical metrics. The effectiveness of the ELM-C1 model over MLR-C1 powerfully appeared in reducing the RMSE and U95 parameters to 10.05% and 34.07%, respectively. Moreover, the prediction accuracy of ELM-C1 accomplished better outcome when it is compared with the RF-C7 model, where there was a significant improvement in the reduction of RMSE and U95 parameters, which reached 16.36% and 49.02%, respectively.Figure 6
The enhancement percentage of the ELM-C1 predictive model over other models based on RMASE, RRMSE, andU59.The box plot diagram, shown in Figure7, presented the best candidate models, which were employed to predict the average monthlyETo. It can be remarkably noted that the RF models could not perform well as compared to the other approaches such as MLR and ELM. However, the MLR models had the modest performance and poor accuracy as compared to ELM approaches. It can be said that the best performance approach in prediction average monthlyETo is ELM, followed by MLR and RF techniques, consecutively. The ELM-C1 achieved the best estimation accuracy where the median and interquarter range (IQR) were found to be very close to the actual median and IQR.Figure 7
Boxplot diagram presented the superiority of ELM over other comparable models during the testing set.Line graph and scatterplot of predicted and actual average monthlyETo are provided in Figures 8(a) and 8(b) during the testing phase. The scatter plot effectively presents useful visualization information on the diversion between observed and predicted values, and the coefficient of determination R2 showed the determination between them. Based on Figures 8(a) and 8(b), the proposed ELM-C1 has better prediction performance than other comparative models in terms of providing a higher value of R20.915.Figure 8
(a): Observed vs. predicted values of mean monthlyETousing different models: testing set. (b) Observed vs. predicted values of mean monthly ETousing different models: testing set.
(a)(b)Considering the best ELM models, it can be noted that this approach often requires a relatively higher number of input parameters in comparison with RF and MLR approaches. The logical explanation of this phenomenon is that the data size includes 35 locations of different characteristics. Besides, the geographical factors are very important when developing local and robust models based on a dataset collected from several locations.For further assessment, it is vital to examine the ability of the suggested model (ELM-C1) against several predictive models that were proposed in the previous studies to estimateETO over different locations around the world. In this respect, the findings obtained during the test step by using the ELM-C1 model are validated against several predictive models carried out in the literature by some researchers. Khoob [94] conducted a study using ANN to predict ETOin Safiabad station, which is located in Southern Iran. The outcome of this study revealed that the suggested ANN model performed very well when compared with the actual values ofETO. The most significant observation is that the accuracy of the model was very good with a high R2of 0.9135. Moreover, another study was carried out using the ANN technique to predict ETO in the Reynolds Creek Experimental Watershed in Southwestern Idaho, USA [95]. The obtained results showed that the ANN model managed to effectively estimate ETOwith high accuracy of prediction and fewer forecasted error reported R2=0.82. Additionally, another study was conducted in Southeast Asia using different AI models; Wu and Fan [96] introduced a comprehensive study to accurately predict ETO over 14 stations located in China. The researchers employed eight different techniques such as ANFIS, MLP, GRNN, KNEA, SVR, XGBoost, M5Tree, and Mars. These models were developed based on the collected data, which includes precipitation and temperature variables. The outcomes of the study illustrated that the machine learning techniques could provide satisfactory prediction of ETOby using only one variable (temperature). In addition, the estimated ETO values were close to the actual ones, and the reported average R2 indicator was equal to 0.829. On top of that, Izadifar and Elshorbagy [97] employed three data-driven models called MLR, GP, and ANN to estimate ETO over Northern Alberta, Canada. Statistical analyses of the study revealed that the most influencing variables on ETO were net radiation and temperature. Besides, the ANN approached trained Bayesian regularization algorithm presented more accurate results than other comparable models, which recorded the highest accuracy of prediction R2=0.82.Significant attempts were made to increasingly constitute powerful models to accurately estimateETObased on few parameters, which can easily be measured such as temperature. Zhu et al. [69] managed to develop a hybrid model called PSO-ELM to predict ETO over five meteorological stations located at the Northwest of China. The proposed model produced more accurate estimations than other AI models and employed empirical equations. The proposed model (PSO-ELM) were built based on temperature only, in general, and produced fewer forecasted errors and higher correlation with the actual one (mean R2 of 0.8905).Overall, the published studies, as illustrated above, have been carried out to estimateETOover several case studies. Even though researchers developed powerful approaches that attained high accuracy, the approaches mentioned could not be applied to constitute a general model instead of simulation of each case study separately. However, in this current study, a single model (ELM-C1) successfully managed to simulate the ETO based on the data collected from dozens of stations. In addition to the simulation process of 35 stations in one single model, the other interesting feature is that the predictability of the proposed model was recorded high R2>0.91.


## 5. Conclusion

Evapotranspiration is considered as one of the most significant factors in the hydrological cycle. Although the well-known Penman–Monteith equation for computing the evapotranspiration exists, there are some difficulties in accurately calculating some of its parameters such as solar radiation and sensible heat flux into the soil. Therefore, in this study, three different approaches were employed, namely, ELM, RF, and MLR, based on geographical and meteorological parameters for the prediction of the mean monthly evapotranspiration over southern Mediterranean coast of Turkey. Besides, twenty different input combinations assigned according to data collected from 35 meteorological stations were established. Ten efficient statistical parameters have been used to assess the 60 predictive models. The outcomes of this study revealed that the performance of the ELM approach outperformed the MLR and RF models. In addition, the efficiency of the proposed ELM produced less computed errors during the testing set. It is worth mentioning that theU95 measure played a vital role in the selection of the best model accuracy among the established 60 models in this study. The other essential observation that can be drawn is that the RF approach provided the perfect accuracy during the training phase, and when it comes to examining its performances during the testing phase, the predictions’ accuracies were very poor and disappointing. Finally, this study suggested the use of ELM in building local models based on different stations and weather conditions. In addition, a study that covers the whole area of the Mediterranean Sea via one robust model is needed. For further investigation, feature selection approaches can possibly be integrated prior to the predictive learning process to extract the essential input variables for the prediction matrix [98, 99].



---

*Source: 8850243-2021-08-24.xml*