Towards an Italian Corpus for Implicit Object Completion
Authors: Agnese Daffara, Elisabetta Jezek (Università di Pavia, 2023)
Topic: Language Models

Abstract
This study focuses on the creation of an Italian corpus tailored for the task of Implicit Object Completion. In this corpus, each sentence includes a token [MASK] indicating the position of the Object's head, along with the annotation of a Gold Standard filler word. The task of Object completion is approached as a masked word prediction task, which can theoretically be executed by a BERT-based transformer model. In the next phase of the project, this task will be applied to a variety of Italian language models, and their performance will be assessed. The goal of the project is to provide insights into the capabilities and limitations of these models in successfully completing Implicit Objects within different contexts.

Introduction
This paper explores the cognitive ability to comprehend verb-argument structures in sentences, specifically focusing on implicit objects. Even when one argument is implicit, individuals can still understand the overall meaning by leveraging the verb's inherent lexical meaning and surrounding context, as suggested by the Distributional Hypothesis (Harris, 1954; Firth, 1951). In the context of Natural Language Understanding (NLU), artificial intelligence systems must replicate this cognitive ability to reconstruct events and identify semantic participants, including implicit arguments. To achieve this, we propose that the task of Implicit Object Completion can be approached as a masked word completion task, which is well-suited to transformer-based models like BERT (Devlin et al., 2019). This paper outlines the creation of a specialized Italian corpus designed for this purpose, which will also be evaluated in future research. The corpus is based on verbs with Optional Objects—those that can either be implicit or explicit. We divide the corpus into two datasets: an IMPLICIT dataset, which includes sentences with implicit objects, and an EXPLICIT dataset, containing sentences with explicit objects. This distinction allows us to evaluate model performance in two contrasting scenarios: the model's performance when the object is implicit versus when it is explicit. Our hypothesis, influenced by prior research (Ye et al., 2020), suggests that models may perform better when trained on implicit data due to the richer contextual information available. The annotation process for the two datasets differs. In the IMPLICIT dataset, two [MASK] tokens are inserted after the verb or adverb to indicate the position of the missing object, which can be completed by either a single noun or a noun phrase (Determiner + Noun). Each sentence is annotated with a Gold Standard (GS) Noun representing the optimal object head for the [MASK] position. In the EXPLICIT dataset, the explicit object’s nominal head is removed and annotated as the GS. The following example illustrates the annotation: 1. IMPLICIT Dataset: Sentence: Da quel 26 dicembre non vuole più bere [MASK][MASK] né lavarsi. Translation: Since that December 26, they no longer want to drink [MASK][MASK], nor wash themselves. GS Object Head: "acqua" (water) Omission Type: LD (lexical deletion) 2. EXPLICIT Dataset: Sentence: Infilo la fetta velocemente nel sacchetto delle fragole e tiro un sospiro, bevo un [MASK] di caffè. Translation: I slide the slice quickly into the strawberry bag and let out a sigh, I drink a [MASK] of coffee. GS Object Head: "sorso" (sip) Omission Type: - The dataset will be used to assess the performance of various Italian BERT-based models on the Implicit Object Completion task. We believe that this annotated dataset, combined with subsequent evaluation, will significantly contribute to the field of Natural Language Processing (NLP), particularly in understanding how models handle optional objects and implicit argument filling.

Conclusion
This paper discusses the ongoing development of a corpus designed for the task of Implicit Object Completion, containing sentences with both Implicit and Explicit Objects. The corpus is divided into two distinct datasets to allow separate analysis. In the IMPLICIT dataset, the position for the missing noun or noun phrase (NP) is indicated by manually inserting two [MASK] tokens after the verb or adverb. The corresponding Gold Standard (GS) object head is manually added, considering the sentence context and the typicality of the verb-object collocation, which is quantified using the ItTenTen20 corpus via Sketch Engine. The omission type is also annotated as either Pragmatic Defaulting (depending on contextual cues) or Lexical Defaulting (depending on the lexical verbal root). For the EXPLICIT dataset, the object’s nominal head is identified manually and replaced with the [MASK] token, with the head annotated separately. The next phase of the project involves evaluating the performance of various BERT-based models on the task of Implicit Object Completion. Cosine similarity will be used as the evaluation metric, which measures the similarity between the model’s predicted word and the GS object head. A higher cosine similarity indicates a closer semantic match. An example comparison between two models, bert-base-xxl and UmBERTo (umberto-commoncrawl-cased-v1), is provided for a sentence from the EXPLICIT dataset. In this case, bert-base-xxl predicts "bicchiere" (cup), which has a higher cosine similarity to "sorso" (sip) than the UmBERTo model's prediction of "cucchiaino" (teaspoon). Both models recognize the word as related to a container, but fail to match the intended category of quantity. The results from this evaluation are expected to raise theoretical questions regarding the models' ability to identify the semantic category of objects and cluster words within the same domain. Additionally, the omission type annotations will provide insights into the role of context in reconstructing implicit objects. These findings may guide further research into how NLP models handle implicit argument filling. In conclusion, this project contributes to the development of a unique Italian corpus for Implicit Object Completion, offering a valuable resource for evaluating language models. The comparison of different BERT-based models, evaluation through cosine similarity, and exploration of omission types will provide important insights into how models handle implicit arguments and contextual information, advancing research in NLP and semantic understanding.