Identifying Semantic Argument Types in Predication and Copredication Contexts: A Zero-Shot Cross-Lingual Approach (2023)

Abstract: Identifying semantic argument types in predication contexts is not a straightforward task for several reasons, such as inherent polysemy, coercion, and copredication phenomena. In this paper, we train monolingual and multilingual classifiers with a zero-shot cross-lingual approach to identify semantic argument types in predications using pre-trained language models as feature extractors. We train classifiers for different semantic argument types and for both verbal and adjectival predications. Furthermore, we propose a method to detect copredication using these classifiers through identifying the argument semantic type targeted in different predications over the same noun in a sentence. We evaluate the performance of the method on copredication test data with Food•Event nouns for 5 languages.

## introduction this paper is concerned with the question of how to automatically decide which semantic type is targeted in predications over nouns. in our case, the predicate can be a verb or an adjective. this question is particularly interesting in cases where complex type nouns foot_0 are arguments of predications. but even with nouns that, lexically, have only a single type, the predication can target a different type and thereby trigger a coercion in the noun @xcite . examples are given in (1). in both (1-a) as well as (1-b), the respective predicates target one of the two types of a complex type noun (a dinner is inherently both an event and a food item). in (1-c), the noun is a simple type noun (soup is only of type food), and its type is targeted in the predication. the predication in (1-d) involves a coercion since it targets a type that is different from the lexical type of the noun. finally, for complex type nouns, we can have cases where different component types of the same noun are targeted, either by different predicates as in (1-e) or by a single predicate as in (1-f) where book is a physical object and an informational content at the same time and the predicate targets both. the first case is an instance of copredication (see below). our main goal is to develop classifiers that, given a predicate and an argument noun in their sentential context, decide whether a specific type has been targeted. furthermore, we exploit the cross-lingual transfer potential of multilingual pre-trained language models (lms) in order to apply this task to different languages without the need of labelled data for all of them. one interesting application of such classifiers is the detection of instances of copredication with complex type nouns. copredication is a general term defining a "grammatical construction in which two predicates jointly apply to the same argument" @xcite . we are interested in a specific type of copredication where two predicates that require different semantic types apply to the same noun @xcite @xcite . for example, given the occur-rence of a complex type noun such as dinner in a sentence where we have two (or more) predications over that noun, we want to decide whether these predications target different types, as for instance in (1-e). we will apply the classifiers developed in this paper to this task, using the complex type food•event as a test case. we start by investigating whether it is possible to train classifiers for both verbal and adjectival predications for this purpose using lms (bert @xcite , mbert @xcite , xlm-roberta @xcite ) as feature extractors. in addition, we investigate whether it is possible to train multilingual classifiers with a zero-shot cross-lingual approach by training the classifiers on one language with the extracted embeddings of multilingual language models and applying them to other languages. we train monolingual classifiers for italian using the extracted embeddings of a monolingual bert model foot_2 and the multilingual ones using the embeddings of the multilingual models mbert and xlm-roberta. we start with italian as our source language due to the availability of annotated data in the t-pas (typed predicate argument structures) resource @xcite . we train classifiers for verbal predications for the semantic types human, information, event, artifact, and location and adjectival predications for the semantic types event, artifact, and information. the selection of these types is intended to capture the diversity of the semantic type hierarchy. finally, we apply the verbal and adjectival classifiers for artifact foot_3 and event semantic types to the sentences containing food•event nouns in order to detect certain copredication patterns, as in (1-e), in which a verb and an adjective predicate over the same noun. we evaluate the proposed model on test data for a set of typologically diverse languages; chinese, english, german, italian, and turkish. foot_4 2 related work ## discussion the findings of the previous studies suggest that lms encode information about the selectional preferences of verbs @xcite @xcite and semantic types of nouns @xcite . our study shows that it is possible to exploit this knowledge of lms to train classifiers for the identification of the semantic types targeted by both verbs and adjectives. from a cross-lingual point of view, our results show that it is possible to use the embeddings of the multilingual lms to train classifiers in order to transfer knowledge from one language to another. our results are in line with the previous studies in terms of the performance of individual models. xlm-roberta yields better performance compared to other multilingual lms @xcite @xcite and its performance is comparable to monolingual models @xcite . even though we have limited test data for adjectival predication classifiers, we expect the transfer to work similarly for both types of predications (verbal and adjectival) and the results for english show this is the case. in the copredication detection task, our results show that classifiers that are trained only with data with simple type nouns are not able to disambiguate the meanings of complex type nouns. this is evi-dent in the tendency of false positives (low specificity) with t-pas classifiers. even when both predications target the same semantic type in a sentence, i.e. in negatives, copredication is detected. this is because both semantic type classifiers tend to classify the predications as positive when a complex type noun is involved. however, this tendency is absent with simple type nouns, which is also evident in the specificity scores. we think that this tendency is due to the nature of complex type nouns and how they are represented by lms, which is a topic we intend to investigate in the future. the false positive tendency is overcome by adding more data with complex type nouns and this improves the overall performance which shows that copredication detection is possible with the proposed model. cross-linguistically, the performance on copredication detection shows a similar pattern for all languages and for both monolingual and multilingual classifiers. in the future, we plan to use this method for building a cross-lingual collection of corpus-based copredication instances that includes also other complex types and copredication constructions. ## conclusion in this study, we focused on training classifiers for the identification of the semantic argument types targeted by the predicates in a specific predication context using the extracted embeddings of lms. we trained both monolingual and multilingual classifiers for different semantic types and for both verbal and adjectival predications. the training results for individual classifiers show that it is possible to train classifiers for this purpose using lms and to train multilingual classifiers with zero-shot cross-lingual transfer using multilingual lms . furthermore, we proposed a method to detect copredications using these classifiers and evaluated the method's performance on cross-lingual copredication test data. our results show that copredication detection is a more complicated task. however, the method achieves reasonable scores for all languages and good scores for english. the monolingual verbal predication classification task. see