pwc-archive/papers-with-abstracts huggingface dataset:
Are Cohesive Features Relevant for Text Readability Evaluation?
Authors: Amalia Todirascu, Thomas Fran{\c{c}}ois, Delphine Bernhard, N{\'u}ria Gala, Anne-Laure Ligozat
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12

This paper investigates the effectiveness of 65 cohesion-based variables that are commonly used in the literature as predictive features to assess text readability. We evaluate the efficiency of these variables across narrative and informative texts intended for an audience of L2 French learners. In our experiments, we use a French corpus that has been both manually and automatically annotated as regards to co-reference and anaphoric chains. The efficiency of the 65 variables for readability is analyzed through a correlational analysis and some modelling experiments.
Named Entity Recognition for Linguistic Rapid Response in Low-Resource Languages: Sorani Kurdish and Tajik
Authors: Patrick Littell, Kartik Goyal, David R. Mortensen, Alexa Little, Chris Dyer, Lori Levin
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Humanitarian, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

This paper describes our construction of named-entity recognition (NER) systems in two Western Iranian languages, Sorani Kurdish and Tajik, as a part of a pilot study of {``}Linguistic Rapid Response{''} to potential emergency humanitarian relief situations. In the absence of large annotated corpora, parallel corpora, treebanks, bilingual lexica, etc., we found the following to be effective: exploiting distributional regularities in monolingual data, projecting information across closely related languages, and utilizing human linguist judgments. We show promising results on both a four-month exercise in Sorani and a two-day exercise in Tajik, achieved with minimal annotation costs.
Multilingual Supervision of Semantic Annotation
Authors: Peter Exner, Marcus Klang, Pierre Nugues
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Question Answering, Relation Extraction, Semantic Parsing, Semantic Role Labeling, Text Summarization

In this paper, we investigate the annotation projection of semantic units in a practical setting. Previous approaches have focused on using parallel corpora for semantic transfer. We evaluate an alternative approach using loosely parallel corpora that does not require the corpora to be exact translations of each other. We developed a method that transfers semantic annotations from one language to another using sentences aligned by entities, and we extended it to include alignments by entity-like linguistic units. We conducted our experiments on a large scale using the English, Swedish, and French language editions of Wikipedia. Our results show that the annotation projection using entities in combination with loosely parallel corpora provides a viable approach to extending previous attempts. In addition, it allows the generation of proposition banks upon which semantic parsers can be trained.
Siamese Convolutional Networks for Cognate Identification
Authors: Taraka Rama
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12

In this paper, we present phoneme level Siamese convolutional networks for the task of pair-wise cognate identification. We represent a word as a two-dimensional matrix and employ a siamese convolutional network for learning deep representations. We present siamese architectures that jointly learn phoneme level feature representations and language relatedness from raw words for cognate identification. Compared to previous works, we train and test on larger and realistic datasets; and, show that siamese architectures consistently perform better than traditional linear classifier approach.
Exploring Differential Topic Models for Comparative Summarization of Scientific Papers
Authors: Lei He, Wei Li, Hai Zhuge
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Sentence, Sentiment Analysis, Topic Models

This paper investigates differential topic models (dTM) for summarizing the differences among document groups. Starting from a simple probabilistic generative model, we propose dTM-SAGE that explicitly models the deviations on group-specific word distributions to indicate how words are used differen-tially across different document groups from a background word distribution. It is more effective to capture unique characteristics for comparing document groups. To generate dTM-based comparative summaries, we propose two sentence scoring methods for measuring the sentence discriminative capacity. Experimental results on scientific papers dataset show that our dTM-based comparative summari-zation methods significantly outperform the generic baselines and the state-of-the-art comparative summarization methods under ROUGE metrics.
Bridging the gap between extractive and abstractive summaries: Creation and evaluation of coherent extracts from heterogeneous sources
Authors: Darina Benikova, Margot Mieskes, Christian M. Meyer, Iryna Gurevych
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Document Summarization, Multi-Document Summarization

Coherent extracts are a novel type of summary combining the advantages of manually created abstractive summaries, which are fluent but difficult to evaluate, and low-quality automatically created extractive summaries, which lack coherence and structure. We use a corpus of heterogeneous documents to address the issue that information seekers usually face {--} a variety of different types of information sources. We directly extract information from these, but minimally redact and meaningfully order it to form a coherent text. Our qualitative and quantitative evaluations show that quantitative results are not sufficient to judge the quality of a summary and that other quality criteria, such as coherence, should also be taken into account. We find that our manually created corpus is of high quality and that it has the potential to bridge the gap between reference corpora of abstracts and automatic methods producing extracts. Our corpus is available to the research community for further development.
Predicting sentential semantic compatibility for aggregation in text-to-text generation
Authors: Victor Chenal, Jackie Chi Kit Cheung
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Clustering, Sentence, Sentence Compression, Text Generation, Text Simplification

We examine the task of aggregation in the context of text-to-text generation. We introduce a new aggregation task which frames the process as grouping input sentence fragments into clusters that are to be expressed as a single output sentence. We extract datasets for this task from a corpus using an automatic extraction process. Based on the results of a user study, we develop two gold-standard clusterings and corresponding evaluation methods for each dataset. We present a hierarchical clustering framework for predicting aggregation decisions on this task, which outperforms several baselines and can serve as a reference in future work.
Sequential Clustering and Contextual Importance Measures for Incremental Update Summarization
Authors: Markus Zopf, Eneldo Loza Menc{\'\i}a, Johannes F{\"u}rnkranz
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Clustering

Unexpected events such as accidents, natural disasters and terrorist attacks represent an information situation where it is crucial to give users access to important and non-redundant information as early as possible. Incremental update summarization (IUS) aims at summarizing events which develop over time. In this paper, we propose a combination of sequential clustering and contextual importance measures to identify important sentences in a stream of documents in a timely manner. Sequential clustering is used to cluster similar sentences. The created clusters are scored by a contextual importance measure which identifies important information as well as redundant information. Experiments on the TREC Temporal Summarization 2015 shared task dataset show that our system achieves superior results compared to the best participating systems.
Natural Language Generation through Character-based RNNs with Finite-state Prior Knowledge
Authors: Raghav Goyal, Marc Dymetman, Eric Gaussier
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Attribute, Language Modelling, Machine Translation, Named Entity Recognition (NER), Text Generation

Recently Wen et al. (2015) have proposed a Recurrent Neural Network (RNN) approach to the generation of utterances from dialog acts, and shown that although their model requires less effort to develop than a rule-based system, it is able to improve certain aspects of the utterances, in particular their naturalness. However their system employs generation at the word-level, which requires one to pre-process the data by substituting named entities with placeholders. This pre-processing prevents the model from handling some contextual effects and from managing multiple occurrences of the same attribute. Our approach uses a character-level model, which unlike the word-level model makes it possible to learn to {``}copy{''} information from the dialog act to the target without having to pre-process the input. In order to avoid generating non-words and inventing information not present in the input, we propose a method for incorporating prior knowledge into the RNN in the form of a weighted finite-state automaton over character sequences. Automatic and human evaluations show improved performance over baselines on several evaluation criteria.
A Hybrid Approach to Generation of Missing Abstracts in Biomedical Literature
Authors: Suchet Chachra, Asma Ben Abacha, Sonya Shooshan, Laritza Rodriguez, Dina Demner-Fushman
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Articles, Information Retrieval, Keyphrase Extraction, Natural Language Inference, Retrieval, Text Summarization

Readers usually rely on abstracts to identify relevant medical information from scientific articles. Abstracts are also essential to advanced information retrieval methods. More than 50 thousand scientific publications in PubMed lack author-generated abstracts, and the relevancy judgements for these papers have to be based on their titles alone. In this paper, we propose a hybrid summarization technique that aims to select the most pertinent sentences from articles to generate an extractive summary in lieu of a missing abstract. We combine i) health outcome detection, ii) keyphrase extraction, and iii) textual entailment recognition between sentences. We evaluate our hybrid approach and analyze the improvements of multi-factor summarization over techniques that rely on a single method, using a collection of 295 manually generated reference summaries. The obtained results show that the hybrid approach outperforms the baseline techniques with an improvement of 13{\%} in recall and 4{\%} in F1 score.
Imitation learning for language generation from unaligned data
Authors: Gerasimos Lampouras, Andreas Vlachos
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Imitation Learning, Structured Prediction, Text Generation

Natural language generation (NLG) is the task of generating natural language from a meaning representation. Current rule-based approaches require domain-specific and manually constructed linguistic resources, while most machine-learning based approaches rely on aligned training data and/or phrase templates. The latter are needed to restrict the search space for the structured prediction task defined by the unaligned datasets. In this work we propose the use of imitation learning for structured prediction which learns an incremental model that handles the large search space by avoiding explicit enumeration of the outputs. We focus on the Locally Optimal Learning to Search framework which allows us to train against non-decomposable loss functions such as the BLEU or ROUGE scores while not assuming gold standard alignments. We evaluate our approach on three datasets using both automatic measures and human judgements and achieve results comparable to the state-of-the-art approaches developed for each of them.
Product Review Summarization by Exploiting Phrase Properties
Authors: Naitong Yu, Minlie Huang, Yuanyuan Shi, Xiaoyan Zhu
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Abstractive Text Summarization, Descriptive, Language Modeling, Language Modelling, Specificity

We propose a phrase-based approach for generating product review summaries. The main idea of our method is to leverage phrase properties to choose a subset of optimal phrases for generating the final summary. Specifically, we exploit two phrase properties, popularity and specificity. Popularity describes how popular the phrase is in the original reviews. Specificity describes how descriptive a phrase is in comparison to generic comments. We formalize the phrase selection procedure as an optimization problem and solve it using integer linear programming (ILP). An aspect-based bigram language model is used for generating the final summary with the selected phrases. Experiments show that our summarizer outperforms the other baselines.
Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts
Authors: Jun Araki, Dheeraj Rajagopal, Sreecharan Sankaranarayanan, Susan Holm, Yukari Yamakawa, Teruko Mitamura
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: coreference-resolution, Coreference Resolution, Multiple-choice, Question Generation, Question-Generation, Reading Comprehension, Sentence

We present a novel approach to automated question generation that improves upon prior work both from a technology perspective and from an assessment perspective. Our system is aimed at engaging language learners by generating multiple-choice questions which utilize specific inference steps over multiple sentences, namely coreference resolution and paraphrase detection. The system also generates correct answers and semantically-motivated phrase-level distractors as answer choices. Evaluation by human annotators indicates that our approach requires a larger number of inference steps, which necessitate deeper semantic understanding of texts than a traditional single-sentence approach.
Evaluation Strategies for Computational Construction Grammars
Authors: T{\^a}nia Marques, Katrien Beuls
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Semantic Parsing

Despite the growing number of Computational Construction Grammar implementations, the field is still lacking evaluation methods to compare grammar fragments across different platforms. Moreover, the hand-crafted nature of most grammars requires profiling tools to understand the complex interactions between constructions of different types. This paper presents a number of evaluation measures, partially based on existing measures in the field of semantic parsing, that are especially relevant for reversible grammar formalisms. The measures are tested on a grammar fragment for European Portuguese clitic placement that is currently under development.
Building a Monolingual Parallel Corpus for Text Simplification Using Sentence Similarity Based on Alignment between Word Embeddings
Authors: Tomoyuki Kajiwara, Mamoru Komachi
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Machine Translation, Sentence, Sentence Similarity, Text Simplification, Translation, Word Embeddings

Methods for text simplification using the framework of statistical machine translation have been extensively studied in recent years. However, building the monolingual parallel corpus necessary for training the model requires costly human annotation. Monolingual parallel corpora for text simplification have therefore been built only for a limited number of languages, such as English and Portuguese. To obviate the need for human annotation, we propose an unsupervised method that automatically builds the monolingual parallel corpus for text simplification using sentence similarity based on word embeddings. For any sentence pair comprising a complex sentence and its simple counterpart, we employ a many-to-one method of aligning each word in the complex sentence with the most similar word in the simple sentence and compute sentence similarity by averaging these word similarities. The experimental results demonstrate the excellent performance of the proposed method in a monolingual parallel corpus construction task for English text simplification. The results also demonstrated the superior accuracy in text simplification that use the framework of statistical machine translation trained using the corpus built by the proposed method to that using the existing corpora.
Broad Twitter Corpus: A Diverse Named Entity Recognition Resource
Authors: Leon Derczynski, Kalina Bontcheva, Ian Roberts
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Diversity, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

One of the main obstacles, hampering method development and comparative evaluation of named entity recognition in social media, is the lack of a sizeable, diverse, high quality annotated corpus, analogous to the CoNLL{'}2003 news dataset. For instance, the biggest Ritter tweet corpus is only 45,000 tokens {--} a mere 15{\%} the size of CoNLL{'}2003. Another major shortcoming is the lack of temporal, geographic, and author diversity. This paper introduces the Broad Twitter Corpus (BTC), which is not only significantly bigger, but sampled across different regions, temporal periods, and types of Twitter users. The gold-standard named entity annotations are made by a combination of NLP experts and crowd workers, which enables us to harness crowd recall while maintaining high quality. We also measure the entity drift observed in our dataset (i.e. how entity representation varies over time), and compare to newswire. The corpus is released openly, including source text and intermediate annotations.
Semantic overfitting: what `world' do we consider when evaluating disambiguation of text?
Authors: Filip Ilievski, Marten Postma, Piek Vossen
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Relation

Semantic text processing faces the challenge of defining the relation between lexical expressions and the world to which they make reference within a period of time. It is unclear whether the current test sets used to evaluate disambiguation tasks are representative for the full complexity considering this time-anchored relation, resulting in semantic overfitting to a specific period and the frequent phenomena within. We conceptualize and formalize a set of metrics which evaluate this complexity of datasets. We provide evidence for their applicability on five different disambiguation tasks. To challenge semantic overfitting of disambiguation systems, we propose a time-based, metric-aware method for developing datasets in a systematic and semi-automated manner, as well as an event-based QA task.
Extraction of Keywords of Novelties From Patent Claims
Authors: Shoko Suzuki, Hiromichi Takatsuka
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Keyword Extraction

There are growing needs for patent analysis using Natural Language Processing (NLP)-based approaches. Although NLP-based approaches can extract various information from patents, there are very few approaches proposed to extract those parts what inventors regard as novel or having an inventive step compared to all existing works ever. To extract such parts is difficult even for human annotators except for well-trained experts. This causes many difficulties in analyzing patents. We propose a novel approach to automatically extract such keywords that relate to novelties or inventive steps from patent claims using the structure of the claims. In addition, we also propose a new framework of evaluating our approach. The experiments show that our approach outperforms the existing keyword extraction methods significantly in many technical fields.
Leveraging Multilingual Training for Limited Resource Event Extraction
Authors: Andrew Hsi, Yiming Yang, Jaime Carbonell, Ruochen Xu
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Dependency Parsing, Event Argument Extraction, Event Extraction, Machine Translation, Named Entity Recognition (NER), Part-Of-Speech Tagging

Event extraction has become one of the most important topics in information extraction, but to date, there is very limited work on leveraging cross-lingual training to boost performance. We propose a new event extraction approach that trains on multiple languages using a combination of both language-dependent and language-independent features, with particular focus on the case where target domain training data is of very limited size. We show empirically that multilingual training can boost performance for the tasks of event trigger extraction and event argument extraction on the Chinese ACE 2005 dataset.
LILI: A Simple Language Independent Approach for Language Identification
Authors: Mohamed Al-Badrashiny, Mona Diab
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Language Identification

We introduce a generic Language Independent Framework for Linguistic Code Switch Point Detection. The system uses characters level 5-grams and word level unigram language models to train a conditional random fields (CRF) model for classifying input words into various languages. We test our proposed framework and compare it to the state-of-the-art published systems on standard data sets from several language pairs: English-Spanish, Nepali-English, English-Hindi, Arabizi (Refers to Arabic written using the Latin/Roman script)-English, Arabic-Engari (Refers to English written using Arabic script), Modern Standard Arabic(MSA)-Egyptian, Levantine-MSA, Gulf-MSA, one more English-Spanish, and one more MSA-EGY. The overall weighted average F-score of each language pair are 96.4{\%}, 97.3{\%}, 98.0{\%}, 97.0{\%}, 98.9{\%}, 86.3{\%}, 88.2{\%}, 90.6{\%}, 95.2{\%}, and 85.0{\%} respectively. The results show that our approach despite its simplicity, either outperforms or performs at comparable levels to state-of-the-art published systems.
High Accuracy Rule-based Question Classification using Question Syntax and Semantics
Authors: Harish Tayyar Madabushi, Mark Lee
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: BIG-bench Machine Learning, Classification, General Classification, Question Answering, Text Classification, Vocal Bursts Intensity Prediction

We present in this paper a purely rule-based system for Question Classification which we divide into two parts: The first is the extraction of relevant words from a question by use of its structure, and the second is the classification of questions based on rules that associate these words to Concepts. We achieve an accuracy of 97.2{\%}, close to a 6 point improvement over the previous State of the Art of 91.6{\%}. Additionally, we believe that machine learning algorithms can be applied on top of this method to further improve accuracy.