Automatic Classification of Attributes in G erman Adjective-Noun Phrases (2021)

Abstract: Adjectives such as heavy (as in heavy rain) and windy (as in windy day) provide possible values for the attributes intensity and climate, respectively. The attributes themselves are not overtly realized and are in this sense implicit. While these attributes can be easily inferred by humans, their automatic classification poses a challenging task for computational models. We present the following contributions: (1) We gain new insights into the attribute selection task for German. More specifically, we develop computational models for this task that are able to generalize to unseen data. Moreover, we show that classification accuracy depends, inter alia, on the degree of polysemy of the lexemes involved, on the generalization potential of the training data and on the degree of semantic transparency of the adjective-noun pairs in question. (2) We provide the first resource for computational and linguistic experiments with German adjective-noun pairs that can be used for attribute selection and related tasks. In order to safeguard against unwelcome memorization effects, we present an automatic data augmentation method based on a lexical resource that can increase the size of the training data to a large extent.

## introduction there is ample evidence that humans decompose the meaning of objects and events into a set of prototypical semantic relations and their values. these relations, referred to in different frameworks as attributes @xcite , frame elements @xcite , thematic relations @xcite , or thematic roles @xcite , serve as an effective means to cluster classes of objects and events by degrees of semantic similarity. for example, thematic roles such as buyer and seller help distinguish among different participants in a financial transaction, and adjectives, such as young and * denotes equal contribution old, group individuals into different equivalence classes for the relation age. likewise, adjectives such as heavy (as in heavy rain) and windy (as in windy day) provide possible values for the attributes intensity and climate, respectively. the attributes themselves are not overtly realized and are in this sense implicit. while these attributes can be easily inferred by humans, their automatic classification poses a challenging task for computational models, as shown in the recent study by @xcite for english data. compared to automatic role assignment for verbal arguments, attribute selection for adjective-noun pairs has received relatively little attention in computational semantics. attribute selection is highly relevant in different nlp tasks, such as information retrieval, topic modelling, and sentiment analysis. consider a sentiment analysis task. if there is positive/negative sentiment expressed about something or someone, it is useful to know what triggers that sentiment. this requires from a system the ability to generalize over specific adjectives to more abstract attributes: attribute: intelligence b. {friendly/rude} people. ## related work earlier studies of attribute selection focus primarily on english data. @xcite and @xcite investigate the attributes in an phrases and create a dataset for english adjective-noun phrases and their corresponding attributes based on the english wordnet. @xcite try to model the task of selecting underlying attributes such as age for a phrase such as old car with representation learning: they experiment with different composition models to construct a single vector for the adjective-noun combination from the embeddings of the adjective and the noun. this composed vector is then used as a proxy for the underlying attribute, e.g. age and ranked with possible alternative values for other candidate attributes. @xcite evaluate different types of word embeddings on a number of lexical semantics tasks, including attribute selection and probe their ability to model lexical composition. for that purpose they reformulate the task of attribute selection into a binary classification: given an adjective-noun pair and an attribute, the classifiers predict whether the target attribute is selected for the pair in question. their findings on the english dataset reveal that this task remains a challenge for all embedding types, though contextualized embeddings clearly outperform static embeddings. our work differs this from previous work in several aspects: we create the first dataset for the annotation of attributes in adjective-noun pairs for german. the taxonomy of 16 attributes is not as finegrained as in @xcite , who distinguishes between 254 attribute labels. our more compact label set is thus more coarse-grained and more suitable for automatic modeling. we test the automatic models in a multiclass-classification setup with the adjective and noun embedding as input. unlike previous work on attribute selection, we take into account whether the semantics of an adjective-noun pair is transparent or not. since the gerco dataset contains both collocations and free phrases, we can partition the data accordingly and can compare the results obtained by a given classifier for the two classes. in earlier work @xcite , we report on binary classifiers for collocational and free adjective-noun pairs, which did not include prediction of the target attributes. in the present paper, the relevant attributes are taken into account. therefore, our research contributes to a growing number of studies of semantic transparency, which up to now have focused on multiword expressions and nominal compounds @xcite @xcite in particular, and extends this body of literature to the empirical domain of adjective-noun pairs. our ability to distinguish between free phrases and collocations, allows us to test the finding of espinosa @xcite , who show that semantic relations in collocations are more difficult to predict in comparison to other types of relations such as hyponymy, meronymy, etc. in sum, previous studies confirm that (i) revealing lexical relations in compounds and an phrases is a challenge in nlp and (ii) relations found in collocations are more difficult to predict than other types of lexical relations. we combine these two findings in our study and model the lexicalsemantic relations, which we call attributes, for both collocations and free phrases. ## data in our experiments, we use the german dataset of adjective-noun phrases gerco @xcite which we annotate with additional seman-tic information. 1 this dataset is suitable for our study due to several reasons: (1) it contains highly polysemous adjectives; (2) half of the dataset is represented by collocations; (3) it is based on a lexical resource -the german wordnet germanet @xcite which can assist us in augmenting the data and obtaining attribute information about it. the original gerco dataset contains 3,652 an phrases manually annotated as "collocations" and "free phrases". the distinction between the two types is based on the transparency of the adjective in the phrase that is operationalized as literality @xcite . for instance, in the phrase grober sand 'coarse sand', the adjective has its literal sense of "rough in texture" -it is annotated as free phrase. in the phrase grober fehler 'gross mistake', the meaning of the adjective is shifted: it does not describe texture in combination with the noun fehler 'mistake', but refers to its intensity. the adjectives in gerco have been chosen on the basis of the semantic classes that they are assigned to in germanet. the advantage of germanet as a lexical resource is that, in contrast to the english wordnet, it models adjectives in a hierarchical manner similarly to nouns and verbs. from each of the 16 semantic classes for german adjectives, three adjectives have been selected. each adjective is paired with the most frequent co-occurring nouns, thus all adjective-noun pairs in the dataset have a strong association. 2 in the present study, we excluded two relational adjectives from the data: barock 'baroque' and steinig 'stony'. out of the remaining 46 adjectives, 44 have at least two senses @xcite . the top nodes of the germanet hierarchy of adjectives represent the 16 semantic classes and the direct hyponyms of the top nodes represent more fine-grained classes of adjectives. 3 figure 1 shows a part of the taxonomy for one sense of adjectives tief 'deep' and salzig 'salty'. the top nodes are used as attribute labels to annotate the data (see section 3.1). we make use of this hierarchical structure for adjectives in germanet in two ways: extracting attribute information (subsection 3.1) and automatic augmentation of the dataset (subsection 3.2). ## automatic classification in the following experiment, we investigate to what extent attribute-selection can be computationally modeled. for that purpose, we use the data described in section 3.3 and train a simple neural network to predict one of the 16 possible attributes given the adjective and noun as input. ## conclusion and future work in this paper we present a study on attribute selection in german adjective-noun phrases. experiments in different training settings with and without lexical overlap show that it is possible to learn attribute selection patterns based on semantically related adjectives and nouns: abstract attributes such as perception, time, or society can be learned and predicted for new, unseen data. the results of the experiments with different lexical overlap settings are in line with previous research: partial lexical overlap leads to better results on this task. however, this is not only due to lexical memorization. the models are still able to decide which attribute to select for an ambiguous adjective in the test set if it appears in training with all its possible meanings, based on the nouns combined with. the experiments confirm that attributes are more difficult to predict for collocations than for free phrases. however, not all types of collocations are equally difficult. attributes can be learned correctly for collocations when the meaning shift occurs systematically. strongly lexicalized collocations cannot benefit from these regularities. as future work it would be interesting to investigate attribute-selection in other languages, e.g., in russian. compounding in russian is not as productive as in german and the function of compounds is often taken over by adjective-noun phrases, so a higher degree of lexicalization would be expected. this could result in an even greater difference between collocations and free phrases. secondly, it would be interesting to investigate how using a full sentence as context impacts the results, especially in ambiguous cases. for instance, the phrase stürmischer tag 'stormy day' can either express the attribute climate when the adjective is used in its literal sense or the attribute manner when @xmath0. for such phrases, disambiguation is only possible in context. finally, it would be useful if a model could learn a general intuition about whether a phrase is a collocation or a free phrase and which attributes are selected by an adjective in its literal and collocational senses.