# Instance Segmentation of Shrimp Based on Contrastive Learning
## 3 Methods
### Contrastive Learning on Shrimp Feature Representations  
Benefiting from the progress of contrastive learning [8; 9; 10; 11; 24; 25; 26; 27], it is possible to train custom backbones for downstream tasks without relying on supervised information. In this paper, we train five state-of-the-art contrastive learning models--MoCov1, MoCov2, MoCov3, SimCLR, and Boyl--as backbones for shrimp instance segmentation. Since the main focus of this paper is on training pre-trained models for shrimp, this subsection only provides a brief introduction to these five models. For more information, readers can refer to the original papers. MoCo series contrastive learning models are proposed in [8; 9; 10]. The main idea of MoCov1 [8] is to build a dynamic dictionary with a queue and a moving-averaged encoder and utilize contrastive loss to update the network. There are two encoders in MoCo architecture, where one encoder named \(q\) is used to extract features of the current query image, while another encoder named \(k\) is employed to obtain the features of the images in the dictionary constructed by the rest of the images in one mini-batch. For one image **I** in a mini-batch, this image is randomly transformed by two different kinds of image augmentations, generating two images: \(\textbf{I}_{q}\) and \(\textbf{I}_{k_{+}}\). \(\textbf{I}_{q}\) is the query image. \(\textbf{I}_{k_{+}}\) is the positive sample of \(\textbf{I}_{q}\) while the rest of the images in this mini-batch are the negative samples of \(\textbf{I}_{q}\). Therefore, there are many negative pairs but only one positive pair in this mini-batch. \(\textbf{I}_{q}\) and \(\textbf{I}_{k_{+}}\) together with other images are sent to the encoders \(q\) and \(k\) to obtain feature maps, respectively. The parameter update of encoder \(q\) is achieved by gradient back-propagation, while the parameter update way of encoder \(k\) is momentum update, which is expressed in the following:  
\[\theta_{k}\longleftarrow m\theta_{k}+(1-m)\theta_{q}. \tag{1}\]  
where \(\theta_{k}\) and \(\theta_{q}\) are the parameters of encoder \(k\) and encoder \(q\), respectively. \(m\in[0,1)\) is a momentum coefficient. In back-propagation, the loss function is the contrastive loss, called NCCLoss, which is  
\[\mathcal{L}_{q}=-log\frac{exp(\textbf{F}_{q}\cdot\textbf{F}_{k_{+}}/\tau)}{ \sum_{i=0}^{k}exp(\textbf{F}_{q}\cdot\textbf{F}_{k_{i}}/\tau)}. \tag{2}\]  
where \(\textbf{F}_{q}\) is the output of encoder \(q\), while \(\textbf{F}_{k_{+}}\) and \(\textbf{F}_{k_{i}}\) are the output of encoder \(k\). \(\tau\) is a temperature hyper-parameter. The value of this loss function becomes small when \(\textbf{I}_{q}\) is similar to its positive sample \(\textbf{I}_{k_{+}}\) and dissimilar to all other negative samples. MoCov2 [9] uses the same architecture as MoCov1 but differs in the methods of image augmentation. MoCov3 [10] primarily utilizes a vision transformer network (ViT) [32] as the encoder for contrastive learning while exploring the performances of different settings in the ViT network. However, since we aim to transfer the visual representations generated by contrastive learning models to shrimp instance segmentation, we replace ViT in MoCov3 with ResNet-50 while keeping the other settings consistent with MoCov3. SimCLR [11] simplifies the contrastive learning architectures by removing the requirement for specialized networks or a memory bank. The differences between SimCLR and MoCo lie in the fact that SimCLR adds a learnable nonlinear transformation between the representation and the contrastive loss to improve the quality of the learned representations. Additionally, SimCLR shows that the composition of multiple data augmentation operations is crucial for contrastive learning, and it thus employs many other kinds of data augmentation. Byol [12] combines the benefits of both MoCo and SimCLR models and achieves a new state-of-the-art performance on many public datasets.