# Improving Sentence Representations via Component Focusing
## 4 Experiment
### Evaluation--Semantic Textual Similarity (STS)  
State-of-the-art methods often learn a complex similarity function that maps sentence embeddings to a similarity score. However, these similarity functions work pair-wise and due to the combinatorial explosion, are often not scalable if the collection of sentences reaches a certain size. Instead, we use cosine similarity to compare the similarity between two sentence embeddings. We evaluated the performance of Universal Sentence Encoder (USE), USE with component focusing, SBERT (first training on NLI, then training on the specific dataset), and CF-BERT (first training on NLI, then training on specific dataset via component focusing) on common STS tasks. We implemented two kinds of CF-BERT based on two pre-trained BERT models, namely, CF-BERTBASE and CF-BERTLARGE. The weight factor \(W_{cf}\) sets the value to 0.2 in the two tasks (described in the next section). We ran our experiments both with Pearson correlation and Spearman correlation as similarity correlation measures. The results are depicted in Table 5. Pearson correlation (_r_) and Spearman \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Task** & **Sentence A** & **Sentence B** & **Output** \\ \hline Sentences Involving & To measure the degree of & A woman with a & The climbing & \\ Compositional & semantic relatedness & & & equipment to rescue a \\ Knowledge Semantic & between sentences from 0 & & wall of rock. & man is hanging from \\ (SICK-R) & (not related) to 5 (related) & & & a white, vertical rock. \\ \hline Sentences Involving & & & & \\ Compositional & To measure semantic in & The dog is snapping & The dog is not & \\ Knowledge & terms of entailment, & at some droplets of & snapping at some & \\ Entailment & contradiction, or neutral & water. & droplets of water. & \\ (SICK-E) & & & & \\ \hline Semantic Textual & To measure the degree of & & & \\ Similarity & semantic similarity & & & \\ Benchmark & between two sentences & & & \\ (STS-B) & from 0 (not similar) to 5 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Downstream tasks description and samples. correlation (\(\rho\)) between the cosine similarity of sentence representations are for two Semantic Textual Similarity (STS) tasks. Performance is reported by convention as \(r\times 100\) and \(\rho\times 100\). The first group of results are the top four SemEval-2014 submissions [40] for SICK-R, the second are more recently proposed methods (including average methods), and the third contains BERT-based methods. The results in black bold represent the best of all methods. Table 5 shows that directly using the output of BERT embeddings leads to rather poor performance even worse than computing average word embeddings (GloVe) sometimes. Using the SBERT (described Siamese network structure and fine-tuning mechanism substantially) improves the correlation, outperforming both Skip-thought and Universal Sentence Encoder. SBERT is slightly weaker than BERT in general. Maybe because by comparing CF-BERT to BERT, there is less interaction of the sentence pairs. The results from Table 5 indicate that all methods with the addition of component focus have improved performance compared to methods without component focusing. USE with component focusing leads to an improvement of 0.7-0.9 points of two tasks. For SICK-R test set, the Pearson correlations of CF-BERTBASE and CF-BERT LARGE are improved by approximately 0.58 and 0.51 points compared to SBERTBASE and SBERT LARGE. The Spearman correlations of CF-BERTBASE and CF-BERT LARGE are improved by approximately 0.51 and 0.42 points compared to SBERTBASE and SBERT LARGE. The results for STS-B test set remained roughly the same. The Pearson correlations lead to an improvement of 0.54 and 0.42 points to CF-BERTBASE and CF-BERT LARGE. The Spearman correlations lead to an improvement of 0.53 and 0.47 points to CF-BERTBASE and CF-BERT LARGE. Our methods easily exceed the first four results.