# High Spatio-Temporal Resolution CYGNSS Soil Moisture Estimates Using Artificial Neural Networks
## 3 Soil Moisture Retrieval Methodology
### ANN Model Architecture  
In this study, we employ a fully-connected ANN architecture, also known as Multilayer Perceptron (MLP), for the nonlinear regression problem, as shown in Figure 2. Input features to the learning process are the surface reflectivity, TES, and SP incidence angle from the CYGNSS observations; NDVI and NDVI-derived VWC from MODIS Aqua Surface Reflectance Daily Global 500m data set [49] to represent the vegetation canopy; elevation, terrain slope, and h-parameter values from the CGIAR-CSI SRTM 90m, Version 4 digital elevation model (DEM) database [50] to stand for the surface dynamics. The selection of these input features were determined after a number investigations on their individual and combined contributions to the estimation performance. The results of these investigations will be provided in Section 4 (see Table 4). The acquisition of the input features from CYGNSS and ancillary data sets as well as the SM data will be comprehensively explained later in this section. Figure 2: Artificial Neural Network with two hidden layers. Input features and the output node are color-coded with respect to separate data sets from which the parameter values were derived. (\(\Gamma_{\textit{bres}}\) is the reflectivity calculated from BRCS and will be defined later.) Reference SM data (in the output layer) are used for optimization of the ANN parameters (minimization of the loss function) in the training stage and assessing the model performance in the validation phase. The proposed model minimizes the loss function, which is defined as the squared error between the model-calculated SM and the reference SM values, over the training data set by running over a predetermined number of iterations. ANN parameters are learned through a stochastic gradient descent solver algorithm, where, within each ANN iteration, the model parameters are updated by computing the partial derivatives of the loss function with respect to the ANN parameters (back-propagation) [51; 52]. In other words, the model learns in the training phase the nonlinear dependences between the CYGNSS measurements and the reference SM labels with the corresponding ancillary data. Then, the trained model uses these dependences to make future SM estimations for a given set of CYGNSS observations and ancillary data. In fully-connected ANNs, neurons of one layer are fully interconnected to each other neuron of the adjacent layer. Each layer has a weights-array that can be trained by the forward and backward propagation mechanisms. This array controls the linear strength of the connections to the next layer [51]. Assuming that the number of neurons in \(i\)th layer is \(N_{i}\), the weights-array at the \(i\)th layer has a size of (\(N_{i+1}\)\(\times\)\(N_{i}\)). The inputs-array has a size of (8 \(\times\) M), where 8 is the number of inputs and M is the number of data samples. The result of the matrix multiplication between weights-array and inputs-array of a particular layer is given as input to the next layer. To account for bias in such a linear relation, a trainable bias value is added to the sum at each neuron. The process described so far defines no more than a linear relation in each neuron, and, if it was the only operation for the entire ANN, it would only result in a linear regression. The essential part of ANNs that make them powerful to solve nonlinear regression problems is the activation function for each neuron. In the literature, a number of different activation functions are used such as Rectified Linear Unit (ReLU), logistic, or tanh activations [53; 54]. These functions are responsible for taking the corresponding bias-added sum as input and transferring it to a new value with the help of the corresponding nonlinear relation. This process at each neuron is repeated until the output layer is evaluated, which gives the predicted SM value in this study. The calculation from inputs to the output is named forward propagation.