## 4 Experiments
### Experimental Setting
#### 4.1.4 Hyper Parameters  
In the experiment of yeast cell counting, the purpose of image segmentation is to determine whether a pixel is a foreground (yeast cell) or background. The last part in the proposed PID-Net before the output is Softmax, which is applied to calculate the classification result of feature maps. The definition of Softmax is shown as Equation (1). \[\text{Softmax}(z_{i})=\frac{e^{z_{i}}}{\sum_{c=1}^{C}e^{z_{c}}}. \tag{1}\]  
In Equation (1), \(z_{i}\) is the output in the \(i\)th node, and \(C\) is the number of output nodes, representing the number of classified categories. The classification prediction can be converted into the probabilities by using the Softmax function, which distributes in the range of \([0,1]\), and the sum of probability is 1. As the image segmentation for yeast counting is to distinguish the foreground and the background. Hence, it is a binary classification, Equation (1) can be rewritten as Equation (2). \[\text{Softmax}(z_{1})=\frac{e^{z_{1}}}{e^{z_{1}}+e^{z_{2}}}=\frac{1}{1+e^{-(z _{1}-z_{2})}}=\text{Sigmoid}(\beta). \tag{2}\]  
In Equation (2), \(\beta\) is \((z_{1}-z_{2})\), which means the Softmax function and the Sigmoid function are the same for binary classification (a little difference between them is, the number of the fully connected (FC) layer of Softmax is two to distinguish two different categories; however, the number of FC layer of Sigmoid is one, only to judge whether the single pixel is the object to be segmented). The probability of the pixel to be classified as 1 is:  
\[\hat{y}=P(y=1|x). \tag{3}\]  
Figure 4: The images in yeast cell dataset. **(a)** The original yeast image and **(b)** the corresponding ground truth images. Apparently, the probability of the pixel to be classified as 0 is:  
\[1-\hat{y}=P(y=0|x). \tag{4}\]  
According to the maximum likelihood formula, the joint probability can be expressed as:  
\[P(y|x)=\hat{y}^{y}\cdot(1-\hat{y})^{1-y}. \tag{5}\]  
After that, \(log\) function is applied to remain the monotonicity invariance of the function:  
\[logP(y|x)=log(\hat{y}^{y}\cdot(1-\hat{y})^{1-y})=ylog\hat{y}+(1-y)log(1-\hat{y }). \tag{6}\]  
Hereto, the loss can be expressed as \(-logP(y|x)\), and the loss function for multiple samples can be defined as the cross-entropy loss (\(N\) is the number of categories):  
\[Loss=-[\sum_{i=1}^{N}y^{(i)}log\hat{y}^{(i)}+(1-y^{(i)})log(1-\hat{y}^{(i)})]. \tag{7}\]  
In order to guarantee the stable and fast convergence of the proposed network, we deploy preliminary experiments to determine the choices of hyper parameters. Adaptive moment estimation (Adam) is compared with stochastic gradient descent (SGD) and natural gradient descent (NGD). Adam optimizer has the smoothest loss curves and stablest convergence, which performs best in microorganism-counting task. Adam optimizer is applied to minimize the loss function, which can adjust the learning rate automatically by considering the gradient momentum of the previous time steps [78]. The initial learning rate is set from 0.0001 to 0.01 in preliminary experiments. By observing the loss curves while training, the learning rate of 0.001 can balance the speed and stability of convergence. The batch size is set as 8 due to the limited memory size (8 GB). The selection of the hyper parameters above are optimal in preliminary experiments, and thus they are applied in our formal microorganism counting experiment. The epoch is set as 100 by considering the converge speed of experimental models, the example of loss and intersection over union (IoU) curves of models is shown in Figure 5.  
Though there are 92,319,298 params to be trained in PID-Net; however, it can converge rapidly and smoothly without over fitting. There is a jump in loss and IoU plots for all three tested networks from 20 to 80 epochs, which is caused by the small batch size. Small batch size may lead to huge difference between each batch, and the loss and IoU curves may jump with convergence. Figure 5: The IoU (**top row**) and loss (**bottom row**) curves in the training process.