# Compositional Distributional Semantics with Syntactic Dependencies and Sectional Preferences
## 7 Conclusions  
In this article, we described a fully compositional strategy based on static distributional models, syntactic dependencies and selectional preferences modeled as paradigmatic relations, which turned out to be competitive when compared to several configurations of BERT in three languages. The fact that compositional models are competitive is a very important factor as they use far fewer parameters in model training than Transformers.  
It is important to note that the proposed distributional model is not based on neural networks and results in transparent vectors, which will facilitate explainability in future work. Transparency enables explainability because it helps to track and select the most relevant linguistic features within the compositional vectors.  
It is also worth noticing that the proposed compositional strategy is not based on Categorial Grammar and typed functional application as other similar works on distributional compositional semantics [15, 20]. In our model, there are no function words like verbs or adjectives, represented as n-order tensors applied on nominal vectors. Function application is driven by binary syntactic dependencies in a semantic space without syntactic constituents. The semantic space proposed in this paper is actually mapped to the Dependency Grammar structure [5].  
However, our method has several weaknesses. Among them, it is too dependent on syntactic parsing and, therefore, it has a vulnerable exposure to parser errors. In addition, another notable weakness is the difficulty in processing open text with all kinds of syntactic constructions.  
In future work, we will study the possibility of designing and developing a model with fully contextualized vectors for open sentences with any syntactic structure. This will be done by defining a more generic concept of selectional preferences, applicable to any dependency tree in an iterative and incremental process of co-selection. For this purpose, we will also consider the compositional meaning of determiners [40], auxiliary verbs, or tense affixes, which are quantifiational operators with different compositional behaviour as lexical units.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Models** & \(\rho\) **(en)** \\ \hline lemma\_SBERT—BERT-large & 61 \\ lemma\_SBERT—BERT-base & 51 \\ lemma\_SBERT—DistilBERT-base & 38 \\ lemma\_SBERT—RoBERTa-base & 47 \\ \hline
**Models** & \(\rho\) **(en)** \\ \hline full\_SBERT—BERT-large & 55 \\ full\_SBERT—BERT-base & 50 \\ full\_SBERT—DistilBERT-base & 41 \\ full\_SBERT—RoBERTa-base & 53 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Spearman correlation between different models of SBERT and human judgments for English. Top: scores obtained using lemma triples (original dataset). Bottom: scores with full sentences derived from the triples.  
Funding: This work has received financial support from DOMINO project (PGC2018-102041-B-I00, MCIU/ AEI/FEDER, UE), eRisk project (RTI2018-093336-B-C21), the Conselleria de Cultura, Educacion e Ordenacion Universitaria (accreditation 2016-2019, ED431G/08, Groups of Reference: ED431C 2020/21, and ERDF 2014-2020: Call ED431G 2019/04) and the European Regional Development Fund (ERDF). Data Availability Statement: Software and datasets are available in a publicly accessible repository: [[https://github.com/gamallo/DepFunc](https://github.com/gamallo/DepFunc)]([https://github.com/gamallo/DepFunc](https://github.com/gamallo/DepFunc)) (accessed on 15 June 2021). Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.