Relational Sentence Embedding for Flexible Semantic Matching
Year: 2023

Abstract:
We present Relational Sentence Embedding (RSE), a new paradigm to further discover the potential of sentence embeddings. Prior work mainly models the similarity between sentences based on their embedding distance. Because of the complex semantic meanings conveyed, sentence pairs can have various relation types, including but not limited to entailment, paraphrasing, and question-answer. It poses challenges to existing embedding methods to capture such relational information. We handle the problem by learning associated relational embeddings. Specifically, a relation-wise translation operation is applied to the source sentence to infer the corresponding target sentence with a pre-trained Siamese-based encoder. Later, the fine-grained relational similarity scores can be estimated with learned relational embeddings. We benchmark our method on 19 datasets covering a wide range of tasks, including semantic textual similarity, transfer, and domain-specific tasks. Experimental results show that our method is effective and flexible in modelling sentence relations and outperforms a series of state-of-the-art sentence embedding methods.1

heading: 1 Introduction
text: Sentence representation learning is a long-standing topic in natural language understanding and has broad applications in many tasks, including retrieval, clustering, and classification (Cer et al., 2018). It has been studied extensively in existing literature (Kiros et al., 2015; Conneau et al., 2017; Reimers and Gurevych, 2019; Gao et al., 2021b) and the problem is defined as learning vector representation (ei) for sentence (si) and the similarity between sentences (si, sj) is inferred by their pairwise cosine similarity: cos(ei, ej). Prior work found that the entailment relation in natural language inference (NLI) data aligns well with the
1Our code is available https://github.com/ BinWang28/RSE
semantic similarity concept defined in STS datasets (Conneau et al., 2017; Reimers and Gurevych, 2019; Gao et al., 2021b). However, the concept of similarity is vaguely defined, and the relationship between sentences is indeed multifaceted (Wang et al., 2022). One can only rely on experimental verification to determine what relation between sentences aligns well with human perception of “semantic similarity”. The similarity intensity between sentences varies greatly based on the perspective of personal understanding.
In this work, we do not focus on discussing the detailed definition of semantic relations but focus on the explicit modelling of sentence relations. Instead of treating sentence pairs as either positive or negative samples in contrastive learning (Gao et al., 2021b; Yan et al., 2021; Wu et al., 2022), we treat sentence pairs as triples: (si, relationship, sj), with relational information embedded. We learn a translational operation between sentence embeddings to contrastively model the relationship between sentences. Despite its simplicity, the re-
238
lational modelling component largely expands the concept of existing sentence embedding and introduces unique properties for more flexible semantic matching capabilities. Our method is called Relational Sentence Embedding (RSE).
First, with relation modelling, RSE can leverage multi-source relational data for improved generalizability. Earlier work mainly focuses on using NLI data for supervised sentence embedding learning, as the best performance is witnessed. It remains an under-explored problem to learn from multi-source relational data jointly for sentence representation learning. Table 1 shows our experimental study to combine multi-source relational data as the positive samples for contrastive sentence embedding learning (Gao et al., 2021b). Simply merging all relational data leads to performance down-gradation (ref. ’All-Above’). In contrast, with explicit relation modelling by translation operations, RSE can effectively leverage multi-source relational information into a shared sentence encoder while providing exceptional performance.
Second, unlike conventional SE, RSE can provide fine-grained relational similarity scores of sentence pairs. As shown in Figure 1b, conventional sentence embedding computes one unified similarity score for sentence pairs. It is less flexible and interpretable as no detailed information is provided about what aspects of sentences are similar. In contrast, as shown in Figure 1c, RSE can infer fine-grained sentence similarity scores according to the relation of interest, which are critical features for heterogeneous applications (e.g., retrieval and clustering). RSE can vastly extend the sentence embedding concept and has excellent potential in methodology development and applications.
We benchmark our approach on 19 datasets to verify the effectiveness and generalizability of our approach. For semantic textual similarity, we con-
duct experiment on seven standard STS (Agirre et al., 2012, 2013, 2014, 2015, 2016; Cer et al., 2017; Marelli et al., 2014) datasets and STR dataset (Abdalla et al., 2021). RSE achieves an averaged Spearsman’s correlation of 82.18% with BERTbase model on STS datasets. We also achieve SOTA performance on 7 transfer tasks (Conneau and Kiela, 2018), and 4 domain-specific tasks (Wang et al., 2021b). We also present the case study and indepth discussions.

heading: 2 Related Work
text: Sentence embedding can be divided into unsupervised and supervised approaches. Even though unsupervised methods have greatly improved with effective data augmentation techniques for contrastive learning (Gao et al., 2021b; Wu et al., 2022; Chen et al., 2022), supervised methods still outperform their unsupervised counterparts. The success of supervised sentence embedding is mainly because of the availability of high-quality natural language inference (NLI) data (Bowman et al., 2015; Williams et al., 2018). InferSent (Conneau et al., 2017), SBERT (Reimers and Gurevych, 2019), and SimCSE (Gao et al., 2021b) are three milestones on sentence embedding that effectively leverage NLI data using LSTM, Pre-trained BERT and contrastive learning, respectively. The NLI data covers three sentence relations: entailment, contradiction and neutral. The relationships between sentences are manifold. However, prior work found that training with other supervised signals (Young et al., 2014; Wieting and Gimpel, 2018) does not achieve comparable performance as NLI data (Gao et al., 2021b). In this work, we focus on learning from multi-source relational signals for better sentence representation learning.
Knowledge graph embeddings are designed for relational data modelling on triples: (h, r, t). The
methods can be classified into structural-based models and semantic-enhanced models. The structural-based methods learn relational embedding and entity embedding jointly through scoring functions like TransE (Bordes et al., 2013), DistMult (Yang et al., 2015) and RotatE (Sun et al., 2019). Semantic-enhanced models incorporate additional information for entities, including entity types and textual descriptions, when learning embeddings. TEKE (Wang et al., 2016), InductivE (Wang et al., 2021a) and KG-BERT (Yao et al., 2019) belong to this type. For sentence representation learning from multi-source supervised signals, the sentence pairs can be viewed as triples: (si, relationship, sj). Therefore, we optimize the semantic representation of sentences and perform translation operations between them to model their respective relations. An analogy to the word-level relational graph as Word-Net (Miller, 1995), and Relational Word Embedding (Camacho-Collados et al., 2019), our work can also be viewed as a pioneering study on the sentence level.

heading: 3 Relational Sentence Embedding
text: Given n pre-defined relations R = {r1, r2, ..., rn}, the multi-source relation sentence pair datasets are composed of triples: D = {(si, rk, sj)}, where si and sj corresponds to the ith and jth sentence. The goal of relation sentence embedding is to learn sentence representations and relational embeddings jointly. The relational similarity intensity between sentences can be inferred from learned embeddings through either simple scoring functions or complex transformations. In this work, as a prototype of RSE, we use translation operation (Bordes et al., 2013) for sentence relation modelling.

heading: 3.1 Relational Contrastive Learning
text: Contrastive learning approaches have demonstrated their effectiveness in sentence embedding learning (Gao et al., 2021b; Janson et al., 2021). The core concept is learning representations that pull together close concepts and push away non-related ones. To build contrastive samples, we leverage the relational signal from sentence triples of supervised data. Assume we have relational triples (si, rk, sj), let hi and hj denote the representation of si and sj obtained from sentence encoder hi = fθ(si) and hrk denote the relational embedding of the kth relation. We then treat hi + hrk and hj as positive pair. Following the previous contrastive learning approach (Chen et al., 2020), other in-batch tail sentences are treated as negatives. Assuming we have N sentence triples in one mini-batch, the training objective can be written as follow:
Li = − log esim(hi+h
r k,hj)/τ
∑N m=1 e sim(hi+h r k,hm)/τ
(1)
where sim(·, ·) is the cosine similarity and τ is the temperature hyperparameter. The pre-trained BERT (Devlin et al., 2019) or RoBERTa (Liu et al., 2019) model fθ(·) is used to obtain sentence embedding. The relation embedding hrk is randomly initialized and jointly optimized with the pre-trained language model with the above relational contrastive objective.
Figure 2 is a graphical illustration of Eq. 1. The relational information is learned by relational embedding through translation operation. That is, for triple (si, rk, sj), we expect hi + hrk ≈ hj. The embedding property can further infer the relation intensity between sentences during inference.
In knowledge graph embedding, there are more diverse ways to model relations, including sim-
ple scoring functions (TransE (Bordes et al., 2013), TransR (Lin et al., 2015) and RotatE (Sun et al., 2019)), semantic matching methods (DistMult (Yang et al., 2015), TUCKER (Balazevic et al., 2019)) and neural network methods (ConvE (Dettmers et al., 2018)). In this work, we experiment with the simple translational operation for relation modeling for simplicity and interpretability. Other complex relation modeling methods can be explored in future work as they are more expressive in certain relation types, including symmetry, anti-symmetry, inversion, and composition (Sun et al., 2019).

heading: 3.2 Supervised Signals
text: The key to contrastive sentence embedding is building positive and negative samples. ConSERT (Yan et al., 2021) proposed to use adversarial attach, token shuffling, feature cutoff, and dropout as positive pair augmentation for unsupervised sentence embedding. SimCSE (Gao et al., 2021b) further explores the potential of dropout as positive-view augmentation. In this work, we collect multi-source relational data as positive samples and view the rest in-batch samples as negatives. In general, multisource relational data can be either supervised or unsupervised. For unsupervised learning, we can view each data augmentation method as one relation and jointly learn from multiple data augmentation methods. In this work, we focus on artificial sentence triples and explore how to combine multiple human-labelled datasets with different relation types.
We deploy the following datasets for sentence triples (si, rk, sj). 1) SNLI (Bowman et al., 2015) + MNLI (Williams et al., 2018): contain natural language inference data. The relations are entailment (r1), contradiction and neutral. We use entailment relation with contradiction relation as hard negatives. 2) QQP2: The Quora Question dataset. The label for each question pair is either duplicationquestion (r2) or non-duplicate-question. We include the triples with duplication-question relation. 3) ParaNMT (Wieting and Gimpel, 2018): paraphrase (r3) dataset generated by back-translation with filtering. 4) Flicker (Young et al., 2014): image captioning dataset. Each image is with 5 human-written captions. We treat the captions for the same image as relation same-caption (r4).
2https://data.quora.com/ First-Quora-Dataset-Release-Question-Pairs
5) QNLI (Wang et al., 2018): contains questionanswer pairs. The label is qa-entailment (r5) or qa-not-entailment based on whether the sentences contain the true answer to the question. We only use qa-entailment triples. 6) Dropout (Gao et al., 2021b): is constructed in an unsupervised way. We use the Wiki-1M dataset and treat the sentence with different random dropouts as one relation. We call this relation same-sent (r6). We leverage all samples from the NLI dataset and at most 150K samples for other relations as the size of QQP for easy comparison.

heading: 3.3 Mining Hard Negatives
text: Contrastive learning commonly uses in-batch sentences as negative samples (Janson et al., 2021). However, the in-batch negatives are limited in relational sentence embeddings. It is because the in-batch samples are coming from different relation triples. The sentences from other relations are less likely to be confused with the current anchor triple. Therefore, we extend the (si, rk, sj) triple to (si, rk, sj , s−j ) with an extra in-relation negative sample s−j . The s − j is one random sampled sentence from the triple of relation rk. To better understand, the in-relation negative for relation duplicate-question will be a random question, a ‘harder’ negative than in-batch sentences. For the entailment relation, we use contradiction sentence as the hard negative. With the hard negative, the training objective can be written as follows (Ref. to Eq.1):
− log e sim(hi+h
r k,hj)/τ
N∑ m=1 (esim(hi+h r k,hm)/τ + esim(hi+h r k,h − m)/τ )
(2) As multiple relation triples are mixed in each batch during training, the batch size for each relation becomes smaller. Contrastive learning favors a relatively large batch size to reach good performance. Wu et al. (2022) incorporate momentum contrast (He et al., 2020) for better contrastive learning. However, we did not find its contribution to the supervised setting. As the GPU memory size constrains the batch size, we scale up the batch size of contrastive learning by decoupling back-propagation between contrastive learning and decoder using Gradient Cache (Gao et al., 2021a). As a result, we can train BERTbase model with batch size 512 within an 11G GPU memory limit. How can we use? As the relational information is
explicitly embedded in RSE, it becomes very flexible for different applications. Given two sentence (si, sj), we can infer their similarity score under relation rk by f(si, sj , rk) = sim(hi + hrk,hj). For retrieval and semantic similarity, we can use any relational similarity score or weighted sum of different relational scores as the final measure. We apply this in STS and USEB benchmarking tasks. As an additional classifier is trained, we use sentence embedding as the input for transfer tasks. We expect that with relational modelling capability, the result sentence embedding carries richer information.

heading: 4 Experiments on STS Tasks
text: We conduct experiment on 7 semantic textual similarity datasets including STS 12-16 (Agirre et al., 2012, 2013, 2014, 2015, 2016), STS-Benchmark (Cer et al., 2017) and SICK-
Relatedness (Marelli et al., 2014). We also experiment with STR-2022 (Abdalla et al., 2021). A new STS dataset with improved label quality using comparative ranking. For a fair comparison, we report the result with the fully unsupervised setting, Spearman’s correlation and ‘all’ for result aggregation following prior works (Reimers and Gurevych, 2019; Gao et al., 2021b).

heading: 4.1 Settings
text: We initialize our training checkpoint with pretrained BERTbase,large (Devlin et al., 2019) and RoBERTabase,large (Liu et al., 2019). The [CLS] token representation is taken as sentence representation (hi). We compare with both unsupervised and supervised sentence embedding methods, including 1) average GloVe embedding (Pennington et al., 2014), 2) average BERT embedding (Su et al., 2021), 3) CT-BERT (Janson et al., 2021), 4) InferSent (Conneau et al., 2017), 5) USE (Cer et al., 2018), 6) SBERT (Reimers and Gurevych, 2019) and 7) SimCSE (Gao et al., 2021b). For a fair comparison, we did not include T5-based methods to avoid the influence of pre-trained checkpoint (Ni et al., 2022; Chen et al., 2022). We use r1 and r2 as our labeled data with relational weights 1.0 and 0.5, respectively. We train our model for 3 epochs and evaluate every 125 training steps on the development set of STS-B to keep the best-performed checkpoint.
τ 0.001 0.01 0.05 0.1 1.0
Avg. STS 80.74 81.73 82.18 77.00 54.88

heading: 4.2 Main Results
text: volving more relational scores, which we leave for future exploration.

heading: 5 Experiments on USEB Tasks
text: As there are some limitations in domain coverage of STS tasks (Wang et al., 2021b, 2022), we also experiment with Universal Sentence Embedding Benchmark (USEB) proposed in Wang et al. (2021b). It contains four tasks from heterogeneous domains, including Re-Ranking (RR), Information Retrieval (IR) and Paraphrase Identification (PI). The datasets are AskUbuntu (RR) (Lei et al., 2016), CQADupStack (IR) (Hoogeveen et al., 2015), TwitterPara (PI) (Xu et al., 2015; Lan et al., 2017), and SciDocs (RR) (Cohan et al., 2020).

heading: 5.1 Settings
text: Similarly, we experiment with four different pre-trained checkpoints BERTbase,large and RoBERTabase,large. We compare with different benchmarking including unsupervised methods: GloVe, Sent2Vec (Pagliardini et al., 2018), BM25 (Robertson et al., 1995), BERT, unsupervised SimCSE, and supervised methods: SBERT (Reimers and Gurevych, 2019), and SimCSE (Gao et al., 2021b). We did not compare with other in-domain adaption-based methods for fair comparison because RSE is an off-the-shelf method. For training, we include all relational data (r1∼6 in Sec. 3.2) and use the similarity score from one specific relation for each task. We train our model for 3 epochs and evaluate every 1000 training steps on the development set of USEB to keep the best-performed checkpoint.

heading: 5.2 Main Results
text: Table 5 shows the evaluation result on 4 USEB tasks. Among the benchmarking methods, we found that unsupervised approaches outperform supervised approaches in AskUbuntu and CQADupStack tasks. The unsupervised SimCSE and BM25 achieve the best result on AskUbuntu and CQADupStack, respectively. It shows the heterogeneous character of USEB task and the previous supervised methods do not generalize well to various domains.
In contrast, RSE performs the best on 3 of the 4 tasks and achieves the best overall performance. The average result improves from 52.1 to 53.7 with BERTbase model. For AskUbuntu and CQADupStack tasks, we find that the unsupervised SimCSE method achieves the best performance, even outperforming the supervised ones. RSE framework aligns with this finding that the best re-ranking and retrieval results are obtained by the similarity inferred from the ‘same-sent’ (r6) relation. The TwitterPara dataset is for identifying whether two sentences are paraphrased or not. In RSE, we find the ‘paraphrase’ relation (r3) trained on synthetic ParaNMT dataset works the best, which aligns with our intuition. Similarly, better performance is achieved with a larger pre-trained model as weight initialization. The USEB experiments verify the RSE’s capability in modelling multi-source relational sentence-pair data. The relational-aware similarity scores can be flexibly applied to heterogeneous domain tasks. Relation Similarity. We obtained 6 relation em-
beddings in our experiments after training with multi-source relational data. To study the similarity between relations, we compute the cosine similarity between pairwise relational embeddings, and the result is shown in Figure 4. From the results, we can witness the more similar relation pairs are entailment & same-sent, duplicate-question & paraphrase, duplicate-question & same-caption, duplicate-question & same-sent, and paraphrase & same-sent. The similar relations are more or less corresponding to sentence similarity, and we can see that the learned relational embeddings capture the relational information well. The qa-entailment relation models the relationship between question and answer pairs. Therefore, we can see it is indeed less similar to all other relations.

heading: 6 Experiments on Transfer Tasks
text: Besides the semantic similarity, ranking, and retrieval tasks, we also experiment with the following transfer tasks: MR (Pang and Lee, 2005), CR (Hu and Liu, 2004), SUBJ (Pang and Lee, 2004), MPQA (Wiebe et al., 2005), SST (Socher et al., 2013), TREC (Voorhees and Tice, 2000) and MRPC (Dolan and Brockett, 2005). Because the RSE model captures richer information by learning from multi-source relational data, we expect it generalizes better to different classification tasks.

heading: 6.1 Settings
text: Similarly, we experiment with four different initialization checkpoints. The averaged [CLS] token representation from the last five layers is taken as
the sentence representation (hi) (Wang and Kuo, 2020). The sentence encoder is used as a feature extractor. An MLP-based classifier is trained with sentence embeddings as the input. The default setting from the SentEval package is deployed (Conneau and Kiela, 2018).
We leverage the relations of entailment and paraphrase for training. As sentence embedding is used as the input to classifiers, relation embeddings are not incorporated in transfer task experiments. All models are trained for 3 epochs. We evaluate the model on the development set every 500 training steps and keep the best-performed checkpoint for final evaluation. We compare RSE with different competitive methods including InferSent (Conneau et al., 2017), USE (Cer et al., 2018), SBERT (Reimers and Gurevych, 2019), and SimCSE (Gao et al., 2021b).

heading: 6.2 Results and Analysis
text: Table 6 shows the evaluation results on 7 transfer tasks. The RSE model with BERTbase architecture achieves an average score of 88.0, outperforming all benchmarked methods. SBERT outperforms SimCSE in 6 of 7 tasks. We infer that the classification-based training objective makes SBERT more transferable to transfer tasks. Even
though the contrastive learning approach outperforms in semantic similarity tasks, it does not generalize well with additional trainable classifiers. In contrast, RSE is a contrastive-based approach, which already showed superior in semantic similarity tasks. Due to its learning capability from multisource relational data, the learned sentence embeddings have better generalizability than singlesource supervised learning. It leads to better transferability of the sentence embedding obtained from RSE models. Similar to previous findings, the performance of RSE improves with larger and better pre-trained checkpoints as initialization. Even though RSE is mainly designed for flexible semantic matching with multiple relational similarity scores, the learn sentence embedding captures richer semantic meanings for various transfer tasks.

heading: 7 Conclusion and Future Work
text: In this work, we propose RSE, a new sentence embedding paradigm with explicit relation modelling by translation operation. RSE can learn from multisource relational data and provide interpretable relational similarity scores. Experiments on semantic similarity, domain-specific, and transfer tasks show that RSE can outperform SOTA approaches, verifying its flexibility and effectiveness. Relational
sentence embedding can broaden the concept of conventional similarity-based sentence embedding and can be extended for different applications.
For future work, first, with more sentence relational data available, the relational sentence embedding can become more competitive and generalizable to different domains. The collection of sentence-level relational data has great potential. Second, more complex relation modelling methods can be incorporated into handling heterogeneous relations. Besides the translation operation in this work, other relation modelling functions, including rotation and semantic matching, can be further explored. Third, this work mainly focuses on supervised relation signals, and unsupervised relational sentence embedding still needs to be explored.
Limitations
The first limitation of RSE comes from data sparsity. Unlike word-level relational data, the variety of sentences is much larger than words. Therefore, even though we can collect more and more relational data on sentence pairs, it is generally hard to become densely connected between sentences. To alleviate this issue and generate sustainable sentence relational data, we should design automatic tools for sentence relation labelling with human-inthe-loop supervision.
Second, unsupervised sentence embedding also shows decent performance on semantic textual similarity tasks, and the focus is mainly on the design of unsupervised contrastive samples. Therefore, exploring different unsupervised view-augmentation techniques in relational sentence embedding remains an open question. In this work, we only study incorporate dropout augmentation without much discussion on other techniques like pseudolabelling (Chen et al., 2022), word drop, word swap, and word repetition (Yan et al., 2021).
For relational data modelling, complex operations (Sun et al., 2019; Trouillon et al., 2016) for relation modelling are introduced and proven effective in knowledge graph completion. However, it is not straightforward to incorporate complex operations with pre-trained semantic representations. The study of RSE in complex space modelling remains unexplored.

heading: Acknowledgement
text: This work is supported by the Agency for Science, Technology and Research (A*STAR) under
its AME Programmatic Funding Scheme (Project No. A18A2b0046), Singapore, and by National Natural Science Foundation of China (Grant No. 62271432).

heading: A Joint-Learning with Multiple Datasets
text: SimCSE (Gao et al., 2021b) found that the positive sentence pairs extracted from entailment relation are the most effective supervision signals in contrastive sentence representation learning. We first experiment with different supervision as positive sentence pairs. The averaged STS result is reported in Table 1. The result shows that using entailment for positive and contrastive for negatives provides the best performance. It outperforms all other supervision, including QQP, Flicker, ParaNMT, and
QNLI datasets. This explains why only NLI dataset is widely used in previous work for supervised sentence representation learning (Conneau et al., 2017; Reimers and Gurevych, 2019; Gao et al., 2021b).
Another question comes to the stage: Can we simultaneously leverage the supervision from the above-mentioned datasets for better-supervised representation learning? An intuitive way to learn from multi-source relational data is to treat all supervised signals as positive samples in contrastive learning. Therefore, we experiment with merging all supervision data and performing contrastive sentence representation learning following Gao et al. (2021b). The result is shown in Table 1. We can tell that merging all sentence pairs for joint training will hurt the performance (77.67 -> 74.62) of STS datasets compared with a single source for positive sentence pairs.
Based on the above experimental results, we conclude it is not a trivial task to leverage multi-source supervision signals in supervised sentence representation learning. With explicit relation modeling techniques proposed in RSE, we can effectively learn from multi-source relational data and achieve better flexibility and performance in various tasks.
B In-domain Relation Prediction
As the relation sentence embedding is trained in a contrastive manner, the trained model can also be used to predict the validity of new triples. Therefore, we did further experiments to test the model’s performance on link prediction.
For dataset creation, we use data from five different relations, including entailment, duplicatequestion, paraphrase, same-caption, and qaentailment. We did not include the prediction for same-sent relation as the prediction for the same sentence is a trivial task. We use all available data for training. For validation and testing, we ran-
domly sampled 2,000 sentence triples each. We perform link prediction experiments following the literature of knowledge graph completion (Sun et al., 2019). The task is to predict the tail entity (sentence) given the head entity (sentence) and relation. It is a retrieval-based task, and performance is measured by the rank of the valid positive samples. Our experimental result is shown in Table 7. From the result, we can see that the paraphrase relation is the easiest to predict. It is understandable because the sentences from the paraphrased data are very similar. In contrast, we see that the same-caption relation is the hardest to be retrieved. It is because the sentence pairs are collected from the descriptions for the same image. The captions for each image can be generated based on various aspects. The relationship between large number sentence pairs is not very strong. It poses challenges on same-caption relations in link prediction.
0 2 4 6 8 10 12 14 16
SciDocs (60+)
TwitterP. (70+)
CQADup. (10+)
AskU. (50+)
entailment duplicate-question paraphrase same-caption qa-entailment same-sent
Figure 5: Performance on the USEB benchmark (4 datasets) using different relational similarity scores. The model is BERTbase.

heading: C Comparison of Relational Score Selection on USEB Tasks
text: In Section 5, we report the best result on four tasks with the best relational similarity score. We found that different dataset favors different relational similarity score. We look into the detailed performance with all relational similarity scores to further study
the effect. The result is shown in Figure 5. For AskU. dataset, the unsupervised SimCSE shows the best performance. Similarly, we also found that the unsupervised relation (same-sent) demonstrated to be the best relation similarity score to be used in AskU. and CQADup. datasets. TwitterP. dataset is a paraphrase identification task and the similarity score under paraphrase relation in RSE shows the best result. In contrast, as no dataset in USEB focuses on question-answer pair retrieval or ranking, the similarity score from qa-entailment relation does not lead to a good performance. From the above discussion, we can see that different relational similarity scores will apply to different applications. A wise choice of relational score can lead to better performance, proving that our proposed RSE model can be flexible to various tasks with explicit relation modeling capability.

heading: D Hyperparameter Settings
text: Table 8 shows the datasets, relations, examples and their applied tasks in our experiments. We train our model for 3 epochs in all experiments and evaluate every 125, 1000, and 500 steps for best checkpoint selection on STS, USEB, and Transfer tasks, respectively. We select the best model based on its performance on the development set. Due to resource limitations, we perform a small-scale grid search for the model hyperparameter settings. The grid search is on batch size ∈ {128, 256, 512} and learning rate ∈ {1e−5, 3e−5, 5e−5}. The learning rate for relation embedding is set to 1e− 2 in all experiments because they are randomly initialized parameters without pre-training. All experiments take the representation for [CLS] token as sentence representation. The temperature hyperparameter is set to 0.05, and the maximum sentence length is set to 32. We did not use warm-up or weight decay in all experiments.

heading: E Case Study
text: To better understand the property of RSE, we present the case study of four sentence pairs and their inferred relational scores in Table 9. The results show that the RSE model can estimate the most likely relationship between sentence pairs. Because the QA entailment relation is very distinct from other relation types, the question-answering pairs are easily distinguished. The QA sentences show the highest score in QA entailment relation, as indicated by the first two cases. Meantime, be-
cause of the similar property between relations of ‘duplicate question’, ‘paraphrase’ and ‘same sent’, from the third case, we can see highly similar sentence pairs have high relational scores in all three relations. It also aligns with our relation study in Figure 4. The last case shows two entailment sentence pairs, and we can see that RSE successfully presents the highest score as an entailment relation.

heading: Future Work
text: For future work, first, with more sentence relational data available, the relational sentence embedding can become more competitive and generalizable to different domains. The collection of sentence-level relational data has great potential. Second, more complex relation modelling methods can be incorporated into handling heterogeneous relations. Besides the translation operation in this work, other relation modelling functions, including rotation and semantic matching, can be further explored. Third, this work mainly focuses on supervised relation signals, and unsupervised relational sentence embedding still needs to be explored. Limitations
The first limitation of RSE comes from data sparsity. Unlike word-level relational data, the variety of sentences is much larger than words. Therefore, even though we can collect more and more relational data on sentence pairs, it is generally hard to become densely connected between sentences. To alleviate this issue and generate sustainable sentence relational data, we should design automatic tools for sentence relation labelling with human-inthe-loop supervision. Second, unsupervised sentence embedding also shows decent performance on semantic textual similarity tasks, and the focus is mainly on the design of unsupervised contrastive samples. Therefore, exploring different unsupervised view-augmentation techniques in relational sentence embedding remains an open question