# Methods for Mid-Term Forecasting of Crop Export and Production
## 3 Materials and Methods
### Modeling of Wheat Export  
#### 3.2.1 Regression Models  
In our study we define the prediction of wheat export and production as a time-series analysis problem. Three modern neural network architectures were created and tested to solve that problem. The NARX-based model [10] is a feed-forward network (Figure 2a) with time-delay inputs, which has layers with _linear_ and _elu_ activations and predicts the future values for the export and production levels \(Y_{E,P}\) (\(t,t+1,..\)) based on the inputs \(X\) and the output values \(Y_{E,P}\) from the previous time-steps (\(t-1..t-n\)). In the experiments we have considered the last three-year history for each prediction; therefore, \(n=3\). The network should forecast the export \(Y_{E}\) and production \(Y_{P}\) two years ahead, and the iterative \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Set** & **Frequency** & **For** & **Features** \\ \hline \multirow{6}{*}{Macro-economic indicators} & \multirow{6}{*}{Annual} & \multirow{6}{*}{Country} & Trade balance \\ \cline{3-3} \cline{5-5}  & & & GDP \\ \cline{3-3} \cline{5-5}  & & & Inflation (CPI) \\ \cline{3-3} \cline{5-5}  & & & Inflation (PPI) \\ \cline{3-3} \cline{5-5}  & & & Population \\ \cline{3-3} \cline{5-5}  & & & Purchasing power parity (PPP) \\ \cline{3-3} \cline{5-5}  & & & Unemployment rate \\ \hline \multirow{2}{*}{NDVI indicators} & Per season: max, min, median & \multirow{2}{*}{Country} & Normalized difference \\ \cline{3-3} \cline{5-5}  & & & vegetation index \\ \hline \multirow{2}{*}{Sentiment} & \multirow{2}{*}{Annual} & \multirow{2}{*}{Country} & Topic-wise summary tweet \\ \cline{3-3} \cline{5-5}  & & & sentiment for a country \\ \hline \hline \end{tabular}
\end{table} 
Table 1: \(Cont.\)  
Figure 1: The data processing pipeline. prediction can lead to error accumulation. Therefore, we modified the model to return the outputs for the whole two-year period at once. During the experiments we utilized and evaluated different sets of features from the dataset to form the inputs \(X\) with the grid search approach. The inputs \(X\) for each time period contain the features of all the considered countries, because all the trade flows are strongly interrelated. We generate and concatenate simple ad-hoc country embeddings \(X_{country}\) to the inputs to help the model distinguish feature groups related to different countries, then we process them with the standard Tensorflow Embedding layer to generate more compact real-valued embeddings. Trade indicators and production level in a particular country can affect the global wheat market significantly; therefore we use the self-attention mechanism to consider long-term dependencies between features. We also limited the model input by the 15 largest wheat exporters and importers to decrease the model complexity. The data processing algorithm in the network is the following. First, the model processes the input features with the dense network \(d_{01}.d_{0k}\). The network has one layer with four neurons (\(k=4\)) and \(elu\) activation. Then we calculate self-attention and concatenate it with the output of the \(d_{01}.d_{0k}\) layer. After that we apply the global average pooling layer to form the attended average embedding of the input data (_Pooling_). Eventually, we process it with another multi-layer feed-forward network \(d_{1}.d_{m}\) to form the outputs. The network has four layers (\(m=4\)) with alternate _linear_ and \(elu\) activations (except the last layer, which has _relu_ activation to prevent the network from negative outputs) and four neurons in each layer. As it is claimed above, the NARX networks require an accurate tuning of the sliding window (time delay) size, and, often it is not feasible because of hardware or time limitations. In addition, it is difficult to consider spatial and temporal dependences with that architecture. The recurrent networks are free from those drawbacks. Figure 2: _Cont._  
In contrast to the NARX model, the recurrent models utilize all the past observations to make the predictions.