Data Augmentation for Hypernymy Detection (2021)

Abstract: The automatic detection of hypernymy relationships represents a challenging problem in NLP. The successful application of state-of-the-art supervised approaches using distributed representations has generally been impeded by the limited availability of high quality training data. We have developed two novel data augmentation techniques which generate new training examples from existing ones. First, we combine the linguistic principles of hypernym transitivity and intersective modifier-noun composition to generate additional pairs of vectors, such as “small dog - dog” or “small dog - animal”, for which a hypernymy relationship can be assumed. Second, we use generative adversarial networks (GANs) to generate pairs of vectors for which the hypernymy relation can also be assumed. We furthermore present two complementary strategies for extending an existing dataset by leveraging linguistic resources such as WordNet. Using an evaluation across 3 different datasets for hypernymy detection and 2 different vector spaces, we demonstrate that both of the proposed automatic data augmentation and dataset extension strategies substantially improve classifier performance.

## introduction the detection of hypernymy relationships between terms represents a challenging commonsense inference problem and is a major component in recognising paraphrase and textual entailment in larger units of text. consequently, it is important for question-answering, text simplification and automatic summarization. for example, there are lots of cars and vans at the port today. ## related work data augmentation has recently become a very popular research topic in nlp and has successfully been applied in machine translation systems @xcite @xcite @xcite @xcite @xcite , but also for tasks such as relation extraction @xcite ), text classification @xcite , or natural language inference @xcite . most similar to our usage of gans for data augmentation is the proposal of @xcite who leverage a ganbased setup together with wordnet for data augmentation for natural language inference. while to the best of our knowledge this work represents the first application of data augmentation for lexical entailment, a number of alternative approaches have been proposed. most proposals rely on supervised methods for injecting an exter-nal source of knowledge into distributional representations. starting with retro-fitting @xcite , vector-specialization methods modify existing representations to embed desired features @xcite @xcite . ## data augmentation strategies for hypernymy detection data augmentation strategies in nlp can roughly be divided into two categories: linguistically grounded augmentation and artificial augmentation. in the former, which has been the dominant paradigm in nlp, any additional instances that are added to a training set have an actual surface form representation, i.e. the data points correspond to actual words or sentences @xcite @xcite @xcite . the latter adds instances that are fully or partly artificial, meaning they do not correspond to any words or sentences. in this work we propose methods for both categories, data augmentation via distributional composition adds data points grounded in real language to a training set, and data augmentation based on gans infers plausible points in latent space, which however, do not correspond to any real linguistic objects. furthermore, we distinguish between data augmentation and dataset extension, where in the former case we only leverage knowledge from the existing dataset and in the latter case we rely on expanding the training set with additionally mined hyponym-hypernym pairs. below, we discuss two ways of augmenting and two ways of extending a training set. we make use of a cleaned october 2013 wikipedia dump @xcite as reference corpus to determine word and bigram frequencies. distributional composition based augmentation. we take a modified noun as being in a hypernymy relation with the unmodified noun. for example, we treat the pairs fast car, car and car, vehicle as expressing the same semantic relation when the modifier-noun compound is composed with an intersective composition function. we focus on adjective-noun (an) and nounnoun (nn) compounds, extracted from our reference corpus where each an or nn phrase occurred at least 50 times. we filtered pairs with non-subsective adjectives using a wordlist from @xcite 2 . we consider two strategies for automatically constructing positive hyponym-hypernym pairs: simple positive cases such as small dog, dog or fast car, car ; and gapped positive cases that mimic the transitivity of hypernym relations, where we pair the hypernym of an existing hyponymhypernym pair with a compound hyponym. for example if car, vehicle is in the training data, we combine car with one of its modifiers to create the pair fast car, vehicle . we construct negative pairs from the simple positive cases using two strategies: creating compositional co-hyponyms such as fast car, red car , where we keep the head noun fixed and pair it with two different modifiers; and creating perturbed simple positive examples, such as small dog, cat where we select the incorrect hypernym (e.g. cat) from the n most similar nouns to the composed hyponym (e.g. dog). we apply the same methodology to the perturbed gapped positive examples, replacing the correct hypernym with a noun from the top n neighbours of the compositional hyponym's head noun. for example, given a positive pair such as dog, animal , this would result in negative examples such as small dog, vehicle , where the hyponym dog is paired with a modifier and the hypernym animal is replaced with one of its neighbours, in this case, vehicle. in neural word embeddings, an additive composition function approximates the intersection of the corresponding feature spaces @xcite , hence by creating positive pairs such as small dog, dog , we encode the distributional inclusion hypothesis @xcite in the augmentation set. ## experiments we evaluate our models on the datasets weeds @xcite and leds @xcite : well-studied and frequently used benchmarks for the hypernymy detection task @xcite @xcite @xcite . since both datasets use wordnet during construction, this can give rise to a bias in favour of those models that also make use of wordnet. to address this concern, we have created a new entailment dataset, hp4k, that makes use of hearst patterns, and is manually annotated, thereby avoiding the use of wordnet. weeds: the dataset is based on nouns sampled from wordnet where each noun had to occur at least 100 times in wikipedia, and its predominant sense had to account for more than 50% of the occurrences in semcor @xcite . we use the predefined split of @xcite , that avoids any lexical overlap between the training and evaluation sets. the split contains 2012 examples in the training set, evenly balanced between positive and negative hyponym-hypernym pairs, and 2002), using wordnet 3.0. ## analysis the concrete amount of data augmentation, i.e. the number of additional hyponym-hypernym pairs that are added to the training set, represents a tuneable parameter. figure 4 shows the effect of varying amounts of data augmentation for the ff model, using word2vec representations, across all datasets. we note that all amounts of additional augmentation data share the same quality, i.e. it is not the case that a smaller augmentation set consists of "better data" or contains less noise than a larger set. for the weeds and leds datasets, peak performance is typically achieved with smaller amounts of additional data, whereas for the hp4k dataset optimal performance is achieved with larger amounts of augmentation data. one explanation for the different augmentation characteristics of the hp4k dataset in comparison to the other two datasets is its independence of wordnet during the development of the dataset. ## conclusion in nlp, in contrast to computer vision, data augmentation has not been applied as standard due to the apparent lack of universal rules for labelinvariant language transformations. we have considered the problem of hypernymy detection, and proposed two novel techniques for data augmentation. these techniques rely on semantic rules rather than an external knowledge source, and have the potential to generate almost limitless synthetic data for this task. we demonstrate that these techniques perform better than word word2vec neighbours gandalf neighbours composed neighbours sugar refined sugar, cane sugar, cmh rawalpindi, prescribed antipsychotic medication, raw sugar, white sugar, brown sugar, turbinado, cocoa, sugars sugar, mumtaz bhutto, akpeteshie sugar price, sugar industry dog dogs, puppy, pit bull, pooch, cat ellis burks, sniffing glue, microchip implants, pet cat, dog fighting, cat breed, rat terrier liz klekamp, cf rocco baldelli terrier breed ocean sea, oceans, pacific ocean, pacific ocean, heavily vegetated, alaska aleutian, ocean basin, pacific ocean, shallow sea, atlantic ocean, oceanic seagrasses, plutoid sea fish, sea mammal animal animals, animal welfare, dog, pet, animals, pet, hallway feeds, poop scooping, first animal, adult animal, zoo animal, cats panhandle animal welfare different animal, animal organization energy renewable energy, enery, radial velocity measurements, stopped, energy efficiency, solar energy, state energy, electricity, enegy, fossil fuel renewable energy, bicycle advisory, steinkuehler food energy, energy company future work includes leveraging data augmentation for more complex models and the extension of our approach to a multilingual setup as well as domains with a more specialised vocabulary such as healthcare or fashion.