Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics (2024)

Abstract: Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. This provides a natural representation for hypernymy but no guarantee that it can be learnt when FDS models are trained on a corpus. In this paper, we probe into FDS models and study the representations learnt, drawing connections between quantifications, the Distributional Inclusion Hypothesis (DIH), and the variational-autoencoding objective of FDS model training. Using synthetic data sets, we reveal that FDS models learn hypernymy on a restricted class of corpus that strictly follows the DIH. We further introduce a training objective that both enables hypernymy learning under the reverse of the DIH and improves hypernymy detection from real corpora.

## introduction functional distributional semantics (fds; emerson and @xcite suggests that the meaning of a word can be modelled as a truth-conditional function, whose parameters can be learnt using the distributional information in a corpus @xcite . aligning with truth-conditional semantics, functional representations of words are logically more rigorous than vectors (e.g., @xcite @xcite and distributions (e.g., @xcite as concepts are separated from their referents (for a discussion, see @xcite . on top of its theoretical favour, @xcite also demonstrate fds models in action and show that they are very competitive in the semantic tasks of semantic composition and verb disambiguation. hypernymy is also known as lexical entailment. it is formally defined as the subsumption of extensions between two word senses, which can be modelled with truth-conditional functions. although fds provides the basis for embedding hypernymy, it is not obvious whether hypernymy can be learnt by training an fds model on a corpus, and if so, what kind of corpus it can successfully learn from. to acquire hypernymy automatically from a corpus, one way is through the use of distributional information. in this class of methods, hypernymy is learnt in an unsupervised manner given certain hypotheses about the distributional properties of the corpus. one such hypothesis is the distributional inclusion hypothesis (dih; @xcite , which relates lexical entailment of words to the subsumption of the typical contexts they appear with in a corpus. in §2, we first highlight that while existential quantifications support the dih, universal quantifications reverse it. in §3, we discuss how fds can embed hypernymy, formulate our hypothesis that fds learns hypernymy under the dih, and introduce a training objective that handles simple universal quantifications. in §4, using synthetic data sets, we show that fds learns hypernymy under the dih, and under the reverse of dih when the new objective is used. in §5, we show that the new objective encodes word generality and improves hypernymy detection by fds on real corpora. ## distributional inclusion hypothesis and quantifications the distributional inclusion hypothesis (dih) asserts that the typical characteristic features (contexts) of r h are expected to appear with r h if and only if r h is a hypernym of r h . although @xcite report that the dih is largely valid on a real corpus, it is not deemed fully correct in general as a hyponym can appear in exclusive contexts due to collocational @xcite and pragmatic reasons @xcite , and feature inclusion has been found to be selective @xcite . in this section, we describe how quantifications can also be pivotal to the hypothesis. next to each noun is the set of contexts that are applicable to the extension of it and those of its descendants (e.g., all dogs are furry, but not all animals.). ## functional distributional semantics in this section, we introduce functional distributional semantics (fds), discuss hypernymy representation in fds and explain how fds can be adapted to handle quantifications. we follow lo et al. (2023)'s fds implementation which is briefly described here. ## experiments on synthetic data sets testing our hypothesis in §3.5 and the effectiveness of the new objective for universal quantifications in §3.6 requires corpora that strictly follow the dih or rdih, which is impractical for real corpora. therefore, we create a collection of synthetic data sets and perform experiments on them. ## experiments on real data sets seeing how fds performs on restricted synthetic data sets is helpful for understanding models' behaviour but it does not immediately tell us more about hypernymy learning from open classes of sentences. therefore, we perform further experiments using a real corpus and data sets for hypernymy. ## conclusion we have discussed how functional distributional semantics (fds) can provide a truth-conditional representation for hypernymy and demonstrate that it is learnable from the distributional information in a corpus. on synthetic data sets, we confirm that fds learns hypernymy under the distributional inclusion hypothesis (dih), and under the reverse of the dih if the proposed objective for universal quantifications is applied. on real data sets, the proposed objective substantially improves fds performance on hypernymy detection. we hope that this work provides insights into fds models and hypernymy learning from corpora in general.