# A Multiple-Input Neural Network Model for Predicting Cotton Production Quantity: A Case Study
## 4 Proposed Multiple-Input Neural Network Prediction Model  
The main contribution of this research is the development of a prediction model for efficiently predicting cotton yield production, which is based on a multiple-input single-output structure. The motivation behind our approach is to develop a learning system that is capable of efficiently exploiting useful information from mixed features of data, since these kinds of data usually require separate treatment. To this end, each input kind of data is processed and handled in a different and independent way. The architecture of the proposed Multiple-input Neural Network (MNN) model is depicted in Figure 1. It consists of three separate input layers (Input Layer -1-, Input Layer -2-, and Input Layer -3-), each one has as inputs the soil (Table 1), the cultivation management (Table 2), and the yield management features (Table 3), respectively. Each input layer is followed by a hidden layer (Hidden Layer -1-, Hidden Layer -2-, Hidden Layer -3-), which independently processes the input data. It is worth noticing that each hidden layer could be constituted by a classical dense layer or by a more sophisticated recurrent layer such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). Next, the outputs of the three hidden layers are imported and merged by a concatenate layer. This layer is followed by a dense layer and a final output layer of one neuron. An advantage of the proposed architecture is that, although stacking many hidden layers allows a traditional neural network model to analyze and encode a very complex function from the input to the output, it is usually difficult to train such a model due to the vanishing gradient problem, which implies that the convergence of the training process may be degradated. Additionally, each category of mixed  
Figure 1: Proposed architecture of a Multiple-input Neural Network (MNN). features of data are handled independently and subsequently the processed data are merged and further processed. As a result, the proposed architecture offers more flexibility and adaptivity for a low development cost compared to that of a fully connected neural network with two or more hidden layers, which implies that computational effort of the training process is reduced. In the sequel, we present a brief description of the Dense, LSTM, and GRU layers which constitute the main elements of the proposed MNN model. * **Dense layer**: Dense layers [20] constitute the traditional and the most popular choice for composing a hidden layer in a multilayer neural network. A dense layer is composed of neurons which are connected with all neurons in the previous layer. The operations performed by each neuron can be summarized by \[o_{i}=f(W_{i}x+b_{i}),\] where \(o_{i}\) is the output of the \(i\)-neuron of the layer, \(W_{i}\) is the vector of weights, \(x\) is the input vector, \(b_{i}\) is the bias vector, and \(f\) is the activation function. * **LSTM layer**: Long Short-Term Memory (LSTM) layers [21] constitute a special type of Recurrent Neural Networks layers which are characterized by their ability to learn long-term dependencies. Each LSTM unit in the layer is composed of a memory cell and three gates: input, output, and forget. At time \(t\), the input gate \(i_{t}\) and a second gate \(c_{t}^{*}\) modulate the amount of information that are stored into the memory state \(c_{t}\). The forget gate \(f_{t}\) modulates the past information which must be vanished or must be kept on the memory cell at the previous time \(t-1\). Finally, the hidden state \(h_{t}\) constitutes the output of the memory cell and it is calculated using memory state \(c_{t}\) and the output gate \(o_{t}\) which modulates the information used for the output of the memory cell.