Extracting Relations from Italian Wikipedia Using Self-Training
Authors: Lucia Siciliani, Pierluigi Cassotti, Pierpaolo Basile, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro (Università di Bari Aldo Moro, 2021)
Topic: Lexical and Semantic Resources and Analysis

Abstract
In this paper, we describe a supervised approach for extracting relations from Wikipedia. In particular, we exploit a self-training strategy for enriching a small number of manually labeled triples with new self-labeled examples. We integrate the supervised stage in WikiOIE, an existing framework for unsupervised extraction of relations from Wikipedia. We rely on WikiOIE and its unsupervised pipeline for extracting the initial set of unlabelled triples. An evaluation involving different algorithms and parameters proves that self-training helps to improve performance. Finally, we provide a dataset of about three million triples extracted from the Italian version of Wikipedia and perform a preliminary evaluation conducted on a sample dataset, obtaining promising results.

Introduction
The goal of an Open Information Extraction (Open IE) system is to extract relations occurring within a text written in natural language. Each relation is structured in the form of a triple that is composed by three elements i.e. {(arg1;rel;arg2)}.  More specifically, given a relation, arg1 and arg2 can be nouns or phrases, while rel is a phrase that denotes the semantic relation between them. Open IE finds its application in several NLP tasks like Question Answering, Knowledge Graph Acquisition, Knowledge Graph Completion, and Text Summarization. For this reason, Open IE is gaining ever-growing attention as a research topic.  Given the nature of the task, approaches for Open IE are deeply intertwined with the language of the corpora that have to be analyzed.  Due to the availability of English corpora, the majority of the state-of-the-art works are specific for that language.  For what concerns the Italian language, the model proposed by guarasci2020lexicon relies on verbal behavior patterns based upon Lexicon-Grammar features. In a previous work, we proposed WikiOIE <cit.>, a framework in which Open IE methods for the Italian language can be easily developed with the aim of encouraging researchers to conduct further work also for under-represented languages. The first solutions developed in WikiOIE are unsupervised, relying merely on PoS tags patterns and dependency relations.  In <cit.> the triples extracted by WikiOIE underwent a deep error analysis. The error analysis reveals syntactic errors such as missing subject or incomplete object information and semantic errors such as generic subject or relation. In this work, we propose a supervised approach to automatically filter out non-relevant triples provided by WikiOIE and a self-training strategy. Self-training <cit.> works iteratively: a classification model is trained on labeled data, the trained model is used to classify unlabeled data i.e. pseudo-labels, the classification model is retrained on labeled data and high-confident pseudo-labels. Specifically, we manually annotate a small number of triples extracted by WikiOIE. Afterward, the annotated triples are augmented using self-training. Finally, the set of triples obtained through self-training at the previous step is exploited to train a supervised model. The paper is structured as follows: after a brief introduction of state-of-the-art methods for Open IE, Section <ref> provides details about the self-training and the supervised model behind our methodology. Section <ref> reports the results of the evaluation, while Section <ref> closes the paper.

Conclusion
We propose a self-training strategy for implementing a supervised open information extraction system for the Italian version of Wikipedia.Our approach exploits a small set of manually labeled triples for expanding the training set.We integrate this system into WikiOIE, which is a framework for open information extraction on Wikipedia dumps.WikiOIE exploits UDPipe as a tool for processing and annotating the text and can be extended by adding several information extraction approaches.We perform an extensive evaluation for measuring the impact of self-training on the overall classification performance.Results prove that self-training is able to improve the classification performance and help to identify not-relevant triples.Finally, we sampled a subset of extracted triples, evaluated by two experts.The number of relevant triples increases when the self-training strategy is used by also improving the agreement between annotators.As future work, we plan to extend the evaluation to a larger scale study, exploit several learning algorithms, and explore the application of the approach to other languages.