# Improving Sentence Representations via Component Focusing
## 5 Discussion: The Impact of Weight Factor \(W_{cf}\) on Model Performance  
As mentioned before, weight factor \(W_{cf}\) is used to adjust the weight of the component-enhanced part so that the best sentence representation is obtained by Equation (1). This section selects a grid  
\begin{table}
\begin{tabular}{c l c} \hline \hline  & \multicolumn{1}{c}{**Models**} & **SICK-E** \\ \hline \multirow{4}{*}{Top Four SemEval-2014 submissions for SICK-E [40]} & Illinois-LH\_run1 & 84.6 \\  & ECNU\_run1 & 83.6 \\  & UNAL-NLP\_run1 & 83.1 \\  & SemantiKLUE\_run1 & 82.3 \\ \hline \multirow{4}{*}{Recently proposed methods (2015–2018)} & Avg. GloVe embeddings [2] & 74.3 \\  & Avg. BERT embeddings [28; 29] & 78.5 \\ \cline{1-1}  & Universal Sentence Encoder (USE) [26] & 83.5 \\ \cline{1-1}  & USE (with CF \(W_{cf}\) = 0.2) (our) & 84.3 \\ \cline{1-1}  & Skip-thought [23] & 82.3 \\ \cline{1-1}  & Infersent [24] & 86.3 \\ \hline \multirow{4}{*}{BERT-based methods} & BERTBASE [12] & 85.8 \\ \cline{1-1}  & SBERTBASE [31] & 85.8 \\ \cline{1-1}  & CF-BERTBASE (with CF \(W_{cf}\) = 0.2) (our) & 86.3 \\ \cline{1-1} \cline{2-4}  & BERTLARGE [12] & 87.2 \\ \cline{1-1}  & SBERTLARGE [32] & 87.1 \\ \cline{1-1}  & CE-BERTLARGE (with CF \(W_{cf}\) = 0.2) (our) & 87.5 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Test set accuracy for the SICK semantic entailment classification.  
search to study the impact of different weight factors on model performance and describes how to find the \(W_{cf}\) corresponding to the best performance.  
Grid search is used for hyperparametric optimization to improve model performance by optimizing the optimal combination of hyperparameters. We only need to determine one hyperparameter \(W_{cf}\) in this paper. Firstly, a list of values is set for the hyperparameter of \(W_{cf}\), and then the computer goes through each to evaluate the performance and select the best one.  
We first chose \(W_{cf}\in\{\)2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0\(\}\) to study the influence of the weight factor on model performance, then \(W_{cf}\) is limited to the range of 0 (without component focusing) to 1.0, with the increasing rate of 0.1 (\(W_{cf}\in\{\)0, 0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0\(\}\)). We changed \(W_{cf}\) and performed SICK-R, SICK-E, and STS-B on CF-BERTBASE and CF-BERTLARGE. We did not observe a significant difference between CF-BERTBASE and CF-BERTLARGE. Experimental results of CF-BERTLARGE on SICK-R and STS-B are displayed in Figure 3 and Table 7 (only Pearson correlation (*100) is displayed).  
We find that the first experiment brings terrible results. Performance is much worse than the models without component focusing. When \(W_{cf}>6.0\), all the performance degradation exceeds six points (we did not show \(W_{cf}\in\{\)7.0, 8.0,9.0,10.0\(\}\)). This demonstrates that component focus can be used as a means of feature supplement rather than dominance. As shown in Figure 3, when the weight factor \(W_{cf}\) is in the range of 0.1 to 0.5, the score first increases and then decreases, but all of the performance is improved compared to the case of \(W_{cf}=0\). Further increasing the value of \(W_{cf}\) would bring negative feedback. When \(W_{cf}\) exceeds 0