Automatic Construction of a Multi-Document Summarization Corpus

The increasing rate of information on information sites, digital books, etc., makes it difficult for readers to obtain the essential information they need in a short time. Not only is the quantity of information huge, but also the heterogeneity of the data makes it difficult to summarize them automatically because of the styles, the text structures, the level of language used and the forms of presenting the information vary from one domain and genre of text to another. That is why the automatic summary of multiple heterogeneous sources is a very interesting field of research. The text summarization (Nenkova & McKeown, 2011) is the process of shortening a text or a group of documents (multi-document summarization) written about the same topic, to a condensed version of a textual document In general, there are two forms of summarization: the abstractive summarization, which consists in expressing the ideas in a source documents by reformulating them and the extractive summarization, which builds the summary from some extracted sentences of the source document. A summary is useful in order to get relevant, important information about a topic without reading it completely. Most of the existing corpus (group of files stored in electronic form and intended for a specific purpose) for multi-document summarization such as MultiLingb (Habernal, Zayed, & Gurevych, 2016) or issue from e.g. the Document Understanding Conference (DUC)(Over, Dang, & Harman, 2007), are mainly focused on homogeneous data, more prevalently in newspapers. These corpora are built manually and contain too few topics. Therefore, they are too small for a reliable analysis using machine learning. (Zopf, Peyrard, & Eckle-kohler, 2016) Propose a heterogeneous corpus, built manually and comprising 10 different genres of documents. They reverse the summarization process by starting from a summary text and create their corpus in three steps: from a summary, some important fragments of information are first extracted, then they are used to download web pages related to the topic, and finally, these web pages are classified in the 10 different genres they proposed. The goal of this thesis is to automate this process and evaluate its performance. Two approaches, namely the part of speech and the named entities have been used to extract the information nuggets. By comparing them to the manually extracted list, the named entities based approach provided better results when considering their F-Measure values (64.76% > 42.59%). Subsequently, topic-related web pages to the information nuggets were downloaded using the Bing Web Search API. Several classifiers were tested on the training data and SVM was more efficient than the others (accuracy = 61.59 %). The causes of this average accuracy have also been explored. The training data was generated from the corpus on the one hand using the StringToWordVector filter of the WEKA tool and on the other hand by building some features related to the web pages. The classification of the newly downloaded web pages reveals that they are made up of 90% of three document genres, and the other seven genres have been represented at 10%. On average, 12 files were assigned to each summary file and split on average into four documents genres. The automation of this process makes it possible to create the largest heterogeneous corpus for multidocument summarization, and the possibility of using the machine learning to evaluate the training set reliably. The tools used allow to extend this solution in addition to the English language, to three other languages Chinese, German, and Spanish.