## 3 Model Training and Loss Function
### The Loss Function  
The overall loss of Mask R-CNN includes two aspects: the loss of classification and regression operations by RPN and the training loss in the multi-branch predictive network. The total loss \(L_{final}\) can be calculated by the following formula:  
\[L_{final}=L_{RPN}+L_{Mul-Branch}\]  
where, \(L_{RPN}\) represents the training loss of the \(RPN\) and \(L_{Mul-Branch}\) represents the training loss due to the branch structure.  
Figure 6: The process diagram of RPN.  
In the training process of _RPN_, there are two things to perform for the generated anchors. The first one is to measure the probability of whether the anchor is the foreground or the background, and the second is to perform the preliminary coordinate correction for the anchors belonging to the foreground. Therefore, \(L_{RPN}\) includes anchors classification loss (SoftMax Loss) and bounding box regression loss (SmoothL1 Loss). \(L_{RPN}\) is calculated as follows:  
\[L_{RPN}=\frac{1}{N_{cls1}}\sum_{i}L_{cls}(p_{i},p_{i}^{*})+\lambda_{1}\frac{1}{ N_{reg1}}\sum_{i}p_{i}^{*}L_{reg}(t_{i},t_{i}^{*}) \tag{1}\]  
\(L_{Mul\text{-}Branch}\) is the sum of three-branch training losses (SoftMax Loss, SmoothL1 Loss and Mask Loss) in multi-branch predictive networks:  
\[\begin{array}{ll}L_{Mul\text{-}Branch}&=L\big{(}p_{i},p_{i}^{*},t_{i},t_{i} ^{*},s_{i}s_{i}^{*}\big{)}\\ &=\frac{1}{N_{cls2}}\sum_{i}L_{cls}(p_{i},p_{i}^{*})+\lambda_{2}\frac{1}{N_{ reg2}}\sum_{i}p_{i}^{*}L_{reg}\big{(}t_{i},t_{i}^{*}\big{)}+\gamma_{2}\frac{1}{N_{mask }}\sum_{i}L_{mask}\big{(}s_{i},s_{i}^{*}\big{)}\end{array} \tag{2}\]  
In the above formula, the constant \(N\)- represents the number of corresponding anchor points or bounding boxes. The hyperparameters \(\lambda^{*}\) and \(\gamma^{*}\) balance the training losses of the regression and mask branches. Here, \(s^{*}\) and \(s\) respectively represent the mask binary matrices from the prediction and ground-truth label. Classification loss \(L_{cls}\), regression loss \(L_{reg}\) and mask loss \(L_{mask}\) are derived from the following formulas:  
\[L_{cls}=-\log(p_{i}^{*}\times p_{i}) \tag{3}\]  
\[L_{reg}=R(t_{i}-t_{i}^{*}) \tag{4}\]  
\[L_{mask}=-(s^{*}\log(s)+(1-s^{*})\log(1-s)) \tag{5}\]  
where \(p_{i}\) represents the classification probability of anchor \(i\) and \(p_{i}^{*}\) represents the ground-truth label probability of anchor \(i\); the variable \(t_{i}\) represents the difference between the prediction bounding box and the ground-truth label box in four parameter vectors (the horizontal, the vertical coordinate value of the center point in the bounding box and the width and height of the bounding box); and \(t_{i}^{*}\) indicates the difference between the ground-truth label box and the positive anchor, \(s^{*}\) and \(s\) respectively represent the mask binary matrices from the prediction and ground-truth label. \(R\) is a robust loss function (smooth \(L_{1}\)).  
\[\text{smooth}_{L_{1}}(x)=\left\{\begin{array}{ll}0.5x^{2}&\text{if }|x|<1\\ |x|-0.5&\text{otherwise}\end{array}\right. \tag{6}\]  
For each ROI, the mask branch has k*m*m dimension output, that is, k masks with m*m size are coded, and each mask has k categories. Since only tomato in green maturity stage was targeted, the value of k is \(1\). For each pixel, the sigmoid function is used to calculate the relative entropy, and the average binary cross-entropy loss function Lmask is obtained, which can effectively avoid inter-class competition.