# Compositional Distributional Semantics with Syntactic Dependencies and Sectional Preferences
## 6 Discussion  
The analysis of the results presented in the previous section leads us to draw some conclusions about the strategies compared in the experiments. Concerning the compositional method, lemmas are more consistent than lexico-syntactic units. The latter have very irregular behavior when constructing meaning from left-to-right and when using the dependent unit as a representative of the sentence. Only heads from right-to-left work properly. Still in the compositional model, the left-to-right process seems to build less reliable compositional vectors than the right-to-left counterpart, at least in this specific dataset. This might be due to the weak semantic motivation of the selectional preferences involved in the subject dependency of transitive constructions in comparison to the direct object. With respect to the results of the Transformers with lemmas, except SBERT (configured only for English), the rest of the systems generally perform worse than the different configurations of the compositional approach with lemmas. To the best of our knowledge, the score achieved by SBERT is, by far, the highest correlation value reported on this \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Models** & \(\rho\)**(pt)** & \(\rho\)**(es)** & \(\rho\)**(en)** \\ \hline \hline nocomp\_lemma—head & 28 & 29 & 29 \\ \hline lemma\_left-to-right—sentence & 42 & 29 & 33 \\ lemma\_left-to-right—head & 43 & 21 & 35 \\ lemma\_left-to-right—dep-obj & 29 & 26 & 19 \\ lemma\_right-to-left—sentence & 45 & 50 & 43 \\ lemma\_right-to-left—head & 43 & 55 & 35 \\ lemma\_right-to-left—dep-subj & 35 & 35 & 44 \\ \hline \hline \multicolumn{4}{l}{[PERSON] and [PERSON] [37]} & - & - & 28 \\ [PERSON] and [PERSON] [39] & - & - & 42 \\ [PERSON] et al. [38] & - & - & 33 \\ [PERSON] et al. [36] & - & - & 54 \\ \hline \hline \multicolumn{4}{l}{**Models**} \\ \hline lemma+rel\_left-to-right—sentence & \(-6\) & \(30\) & \(19\) \\ lemma+rel\_left-to-right—head & \(-1\) & \(40\) & \(18\) \\ lemma+rel\_left-to-right—dep-obj & \(2\) & \(4\) & \(10\) \\ lemma+rel\_right-to-left—sentence & \(34\) & \(32\) & \(27\) \\ lemma+rel\_right-to-left—head & \(38\) & \(39\) & \(53\) \\ lemma+rel\_right-to-left—dep-subj & \(1\) & \(3\) & \(-13\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Spearman correlation between different configurations of DepFunc and human judgments on subject-verb-object expressions. Correlation was computed on benchmarks for English (en), Portuguese (pt) and Spanish (es). Top: scores with configurations based on vectors of lemmas. Bottom: scores with configurations based on lexico-syntactic units (lemma + rel). \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Models** & \(\rho\)**(pt)** & \(\rho\)**(es)** & \(\rho\)**(en)** \\ \hline \hline lemma\_SBERT—sentence (fine-tuned) & - & - & **61** \\ \hline lemma\_bert—sentence & 32 & 19 & 10 \\ lemma\_bert—head & 27 & 34 & 38 \\ lemma\_bert—dep-subj & 16 & 27 & 9 \\ lemma\_bert—dep-obj & 27 & 32 & 21 \\ \hline \hline
**Models** & \(\rho\)**(pt)** & \(\rho\)**(es)** & \(\rho\)**(en)** \\ \hline full\_SBERT—sentence (fine-tuned) & - & - & 55 \\ \hline full\_bert—sentence & 41 & 28 & 30 \\ full\_bert—head & 43 & 42 & 45 \\ full\_bert—dep-subj & 33 & 30 & 27 \\ full\_bert—dep-obj & 39 & 31 & 36 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Spearman correlation between different configurations of BERT and human judgments. Correlation was computed on benchmarks for English, Portuguese and Spanish. Top: scores obtained using lemma triples (original dataset). Bottom: scores with full sentences derived from the triples. dataset. Our results coincide exactly with those reported in [36] for this system and this dataset. It should be noted, however, that SBERT is a supervised approach as it relies on more than 1 million annotated sentence pairs.