## 1 Introduction  
Semantic Textual Similarity (STS) is one of the fundamental tasks in the Natural Language Processing (NLP) that measures similarity between two texts regardless of having or not having common words. In fact, STS deals with computing the similarity between conceptually similar, but not necessarily lexically similar texts. STS plays a significant role in many NLP applications including information retrieval [1, 2], text summarization [3, 4], text classification [5, 6], sentiment analysis [7, 8], question answering [9, 10], machine translation [11, 12], entity recognition [13], etc. Although recent sentence-encoding models such as BERT [14] have been very successful in some NLP tasks, without fine-tuning, the sentence embeddings generated with them fail to capture all the semantic meaning of sentences in the STS task and have low quality [15], i.e., they are mapped into a small area and so the cosine similarity scores between almost all sentence pairs are in the range of 0.6 and 1.0 [16]. This problem, which is referred to as the collapse issue of BERT in [16, 17], stems from the anisotropic space of representation, as sentence embeddings generated by BERT depend on the frequency of words [15, 16]. Therefore, if two sentences have some words in common, the cosine similarity between them is high regardless of whether they have completely different semantic meanings. For example, consider the following three sentences in BERT representation space: \"A woman is walking across the street eating a banana, while a man is following with his briefcase\", \"A woman eats ice cream walking down the sidewalk, and there is another woman in front of her with a briefcase\", \"A person eating crosses the street.\" The cosine similarity between the first and the second sentences is more than the cosine similarity between the first and the third sentences, while the first and the second sentences have completely different semantic meanings. One of the general strategies to prevent these problems and achieve the desired performance on the STS task is to use contrastive learning for fine-tuning BERT [17]. Contrastive learning is a deep metric learning approach, the main idea of which is to learn such a representation space where similar sample pairs (anchor-positive) are brought closer together, while dissimilar sample pairs (anchor-negative) are pushed further away. Contrastive learning has shown outstanding performance on computer-vision tasks such as human-activity recognition [18, 19, 20, 21, 22, 23], person re-identification [24, 25], object detection [26, 27], image classification [28, 29], image processing [30, 31], etc. Contrastive learning can be used in two ways, self-supervised and supervised settings. The self-supervised contrastive learning contrasts a single anchor-positive pair in a batch against many negative samples (other classes' positive samples in a batch), while supervised contrastive learning contrasts a set of all positive samples from the same class against many negatives (other classes' positive samples in a batch) [32]. Recently, contrastive learning has been used for fine-tuning the BERT pre-trained language model [16, 33, 34, 35, 36, 37]. The method in [16, 33, 34, 35, 36] employs self-supervised NT-Xent loss from SimCLR paper [38] which accepts positives sentence pairs in the form of anchor-positive \((x_{i},x_{i}^{+})\) and uses other input positive pairs as negative examples for contrastive learning. After this, ref. [37] proposed a supervised SimCSEsup model by applying a hard negative to NT-Xent loss which accepts triplets in the form of anchor-positive-negative \((x_{i},x_{i}^{+},x_{i}^{-})\). However, recent literature on contrastive learning has attempted to enhance its discrimination performance by including multiple hard positives [32, 39] or multiple hard negatives [40, 41].