Distributional semantic models provide vector representations for words by gathering co-occurrence frequencies from corpora of text. Compositional distributional models extend these representations from words to phrases and sentences. In categorical compositional distributional semantics these representations are built in such a manner that meanings of phrases and sentences are functions of their grammatical structure and the meanings of the words therein. These models have been applied to reasoning about phrase and sentence level similarity. In this paper, we argue for and prove that these models can also be used to reason about phrase and sentence level entailment. We provide preliminary experimental results on a toy entailment dataset. The distributional hypothesis (Firth 1957) provides a model in which meanings of words are represented by vectors. These vectors are built from frequencies of cooccurrences of words within contexts. Compositional distributional models (Mitchell and Lapata 2010) extend these vector representations from words to phrases/sentences. They work alongside the principle of compositionality, employing the fact that the meaning of a string of words is a function of the meanings of the words therein. The vectorial word and phrase/sentence representations have been applied to similarity-based language tasks such as disambiguation and semantic similarity (Schütze 1998; Turney 2006) . In order to apply the distributional representations to entailment tasks, distributional semanticists adhere to a distributional inclusion hypothesis: if word v entails word w, then the contexts of word v are included in the contexts of word w. This means that whenever word v is used, word w can be used retaining a valid meaning. Whereas there has been an extensive amount of work on this hypothesis at the word level, e.g. see (Dagan, Lee, and Pereira 1999; Weeds, Weir, and McCarthy 2004; Kotlerman et al. 2010) , not much has been done when it comes to phrases/sentences. The work on entailment between quantified noun phrases (Baroni et al. 2012 ) is an exception, but it does not take into account composition. Compositionality is what is needed for a modular approach to the textual entailment challenge (Dagan, Glickman, and Magnini 2006) , where entailment is to be decided for complex sentences of language. * Author order is alphabetical. Categorical compositional distributional semantics (CCDS) is a compositional distributional model where vectorial meanings of phrases/sentences are built from the vectors of the words therein and grammatical structures of the phrases/sentences (Coecke, Sadrzadeh, and Clark 2010). These models offer a general mathematical setting where the meaning of any phrase/sentence of language can in principle be assigned a vectorial representation. Fragments of them have been instantiated to concrete data and have been applied to word and phrase/sentence similarity-based tasks, outperforming the models where grammar was not taken into account (Grefenstette and Sadrzadeh 2011; Kartsaklis and Sadrzadeh 2013) . In this paper we show how CCDS can be used to reason about entailment in a compositional fashion. In particular, we prove how the general compositional procedure of this model extends the entailment relation from words to strings of words whose grammatical structure is obtained from their syntactic parses. Previous work on word level entailment shows how entropy-based notions such as KL-divergence can be used to formalise the distributional inclusion hypothesis (Dagan, Lee, and Pereira 1999; Herbelot and Ganesalingam 2013) . In the current paper we prove that in CCDS this notion soundly extends from word vectors to sentence vectors and provides a notion of sentence entailment similar to that of Natural Logic (MacCartney and Manning 2007) . In the presence of correlations between contexts, the notion of KL-divergence naturally lifts from vectors to density matrices via von Neumann entropy. The results of this paper build on the developments of (Balkır 2014; Balkır, Sadrzadeh, and Coecke 2015); we generalise the instantiations of CCDS from vectors to density matrices and argue that the notion of relative entropy on density matrices gives rise to a richer notion of word and sentence level entailment. Density matrices have been previously used in compositional distributional semantics to represent parsing information (Blacoe, Kashefi, and Lapata 2013) and ambiguity of meaning (Piedeleu et al. 2015) . We conclude by providing a small scale experiment on data obtained from British National Corpus (BNC) applied to a toy short-sentence entailment task. This involves implementing a concrete way of building vectors and density matrices for words and composing them to obtain sentences.
REF proposed compositional distributional models to extend representations from words to sentences.
Sentence Entailment in Compositional Distributional Semantics
Ann Math Artif Intell (2018) 82: 189. https://doi.org/10.1007/s10472-017-9570-x
Computer Science Mathematics