# Methods for Mid-Term Forecasting of Crop Export and Production
## 4 Results  
First, we separated the dataset into the train and validation subsets. We used cross-validation on the train subset to set the hyper-parameters (dropout level, number of cells in the layers, weight of the regularization). Then we utilized bootstrapping to assess the models on the validation subset because most results are quite unstable. We applied a mean squared error \(MSE\) (7) and root relative squared error \(RMSE\) (8) as the validation scores. \[MSE=\frac{1}{N}\sum_{i=1}^{N}(Y_{i}-Y_{i}^{*})^{2} \tag{7}\]  
\[RMSE=\sqrt{\frac{\sum_{i=1}^{N}\big{(}Y_{i}-Y_{i}^{*}\big{)}^{2}}{\sum_{i=1}^ {N}\big{(  
\"ALL--NDVIS\"--all the features above except vegetation indexes. \"ALL--NDVIS--FUEL\"--all the features above except vegetation indexes and fuel prices. It is worth noting that the best prediction scores were obtained for the models trained on the full feature set. That means both the vegetation indexes and fuel prices are crucial for the export prediction, although those overall are quite poor for real applications (\(\text{RMSE}\approx 0.45\)). Table 4 presents the two year production level prediction results. All the results are fairly good, and the Transformer model shows the best prediction accuracy (\(\text{RMSE}\approx 0.14\)). We could not find any score improvement related to the NDVI indicators. We believe this is because the experimental dataset is quite small; therefore adding features leads to worse generalization. Additionally, there is a correlation between past production and vegetation levels, which makes the latter redundant. However, for most applications those vegetation indicators still would be crucial, because they are available almost immediately, while other indicators are collected by statistic agencies with a large gap. Excluding macro-economic indicators has shown minor increase (\(\approx\)0.02-0.03 for RRSE), both for export and production prediction errors. Table 5 shows the two-year export prediction results for the models with stationary term and textual features. The notation for the considered feature sets is the following. \"ALL + SE\"--all the features, models with the stationary term. The table shows that the regularization helps to significantly improve the export prediction accuracy (\(\text{RMSE}\approx 0.22\)). In addition, it reduces the variance of the errors. Additional textual features also improve the predictions, especially for the NARX and Transformer models (\(\text{RMSE}\approx 0.21\)). In contrast to the export prediction models without regularization, those models are accurate enough to be utilized for information support of decision makers, who form state and large company agricultural policy. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Features** & **ALL** & \multicolumn{2}{c}{**ALL**} & \multicolumn{2}{c}{**ALL—NDVIS**} & \multicolumn{2}{c}{**ALL—NDVIS—FUEL**} \\ \hline
**Models** & \(\text{MSE}\)**, \(\times\)\(10^{9}\)** & \(\text{RMSE}\) & \(\text{MSE}\)**, \(\times\)\(10^{9}\)** & \(\text{RMSE}\) & \(\text{MSE}\)**, \(\times\)\(10^{9}\)** & \(\text{RMSE}\) \\ \hline
**GRU** & 1.80 \(\pm\) 0.47 & 0.15 \(\pm\) 0.07 & 1.88 \(\pm\) 0.24 & 0.16 \(\pm\) 0.05 & **1.68 \(\pm\) 0.57** & **0.15 \(\pm\) 0.09** \\ \hline
**LSTM** & 3.76 \(\pm\) 0.41 & 0.22 \(\pm\) 0.07 & 3.08 \(\pm\) 0.36 & 0.20 \(\pm\) 0.07 & 1.69 \(\pm\) 0.37 & 0.15 \(\pm\) 0.07 \\ \hline
**SimpleRNN** & 1.81 \(\pm\) 0.30 & 0.15 \(\pm\) 0.06 & 1.83 \(\pm\) 0.27 & 0.15 \(\pm\) 0.05 & 1.94 \(\pm\) 0.53 & 0.16 \(\pm\) 0.08 \\ \hline
**NARX** & 1.83 \(\pm\) 0.29 & 0.15 \(\pm\) 0.06 & 1.92 \(\pm\) 0.57 & 0.16 \(\pm\) 0.08 & 1.92 \(\pm\) 0.36 & 0.16 \(\pm\) 0.07 \\ \hline
**Transformer** & **1.77 \(\pm\) 0.34** & **0.15 \(\pm\) 0.06** & **1.56 \(\pm\) 0.59** & **0.14 \(\pm\) 0.08** & 1.82 \(\pm\) 0.25 & 0.16 \(\pm\) 0.06 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Production prediction scores for the baseline models.