## 3 Methodology
### ANN Model Training
#### 3.2.2 Transfer Learning  
In this study, two-stage training was adopted and the main stage part was improved by the implementation of fine-tuning. The procedures for the fine-tuning in the proposed transfer learning is as follows. There are two sub-stages in the main training.  
1. Main-training stage. With the help of the pretraining stage, the ANN model contains suitable values of unknown parameters. Training them with the rebalanced dataset increases the possibility that the search for the global minimum in the loss function can be reached. This also means that the accuracy of the estimation is enhanced or the estimation error is smaller. The error is then backpropagated to update the unknown parameters and the spatial feature is more robust.
2. Fine-tuning stage. In the previous stage, the extraction of the spatial feature is already powerful, and continuing training the previously trained ANN model with the rebalanced dataset may only endanger the spatial feature. Therefore, a fine-tuning technique is performed in this stage by means of \"network surgery\". First, the model is split into two parts: the body part, consisting of the first and second phase of the ANN model; and the head part, consisting of the last phase of the ANN model, which is the Chla concentration estimation phase. The head part is then removed, leaving the body part only. Inputting an image patch to the body part only will result in a spatial feature image. In machine learning, the technique to split and remove the head part is known as a feature extractor. Moreover, a new head part containing a similar network as the last phase with a random initial value for the unknown parameters is attached to the body part. Here, if the gradient is allowed to backpropagate from these random values all the way through the network, the powerful spatial features could be at risk. To prevent this problem, the layers in the body part, i.e., in the first and second phase of the model, are frozen or set as untrainable and allow the backpropagation when training be performed on the new head only. This allows the network to start learning from the powerful spatial feature and the estimation of Chla concentration can be optimized. Lastly, all of the layers are unfrozen or set as trainable. However, different to the previous stage or sub-stage in which the training is conducted with a learning rate of 0.001, the learning rate is now set to a very small rate of 0.0001. The aim of setting such very small rate is to obtain a suitable adjustment for the body and head parts. For simplification, Figure 5 shows the workflow of the fine-tuning stage.  
Figure 5: Training flow in the main-training stage of the proposed method. Blue and grey boxes represent trained unknown parameters in a layer with a learning rate of 0.001 and 0.0001, respectively; the green box denotes the random reinitialization of unknown parameters in a layer.  
For hyperparameters, the Adam optimizer is employed due to its capability in adaptively tuning the learning rate and moment [46], and the mean squared error (MSE) is used as the loss function \(L\) and is defined as follows:  
\[L=\frac{1}{m}\sum_{i=1}^{m}(prChla_{i}-isChla_{i})^{2}, \tag{1}\]  
where \(prChla_{i}\) is the prediction or estimation of Chla concentration from the input image patch of the \(i\)-th labelled data. Moreover, overfitting is alleviated by adopting two regularization techniques: dropout and \(L_{2}\) regularization. The dropout rate is set to 0.5, meaning that only 50% of the total unknown parameters are temporarily deactivated when computing the loss function for model convergence monitoring, whereas the \(L_{2}\) regularization adds the Frobenius norm to the loss function to penalize large weights during error backpropagation for the tuning of unknown parameters. The maximum epoch is set to 30 and the trained network from an epoch with the smallest value of the loss function will be stored and used for the Chla concentration estimation.