pwc-archive/papers-with-abstracts huggingface dataset:
Interpretable Rationale Augmented Charge Prediction System
Authors: Xin Jiang, Hai Ye, Zhunchen Luo, Wenhan Chao, Wenjia Ma
Date: 2018-08-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Classification, Deep Reinforcement Learning, General Classification, Prediction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), text-classification, Text Classification

This paper proposes a neural based system to solve the essential interpretability problem existing in text classification, especially in charge prediction task. First, we use a deep reinforcement learning method to extract rationales which mean short, readable and decisive snippets from input text. Then a rationale augmented classification model is proposed to elevate the prediction accuracy. Naturally, the extracted rationales serve as the introspection explanation for the prediction result of the model, enhancing the transparency of the model. Experimental results demonstrate that our system is able to extract readable rationales in a high consistency with manual annotation and is comparable with the attention model in prediction accuracy.
A Cross-lingual Messenger with Keyword Searchable Phrases for the Travel Domain
Authors: Shehroze Khan, Jihyun Kim, Tarik Zulfikarpasic, Peter Chen, Nizar Habash
Date: 2018-08-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Machine Translation, Sentence, Translation

We present Qutr (Query Translator), a smart cross-lingual communication application for the travel domain. Qutr is a real-time messaging app that automatically translates conversations while supporting keyword-to-sentence matching. Qutr relies on querying a database that holds commonly used pre-translated travel-domain phrases and phrase templates in different languages with the use of keywords. The query matching supports paraphrases, incomplete keywords and some input spelling errors. The application addresses common cross-lingual communication issues such as translation accuracy, speed, privacy, and personalization.
Towards Automated Extraction of Business Constraints from Unstructured Regulatory Text
Authors: Rahul Nair, Killian Levacher, Martin Stephenson
Date: 2018-08-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Entity Extraction using GAN, Question Answering

Large organizations spend considerable resources in reviewing regulations and ensuring that their business processes are compliant with the law. To make compliance workflows more efficient and responsive, we present a system for machine-driven annotations of legal documents. A set of natural language processing pipelines are designed and aimed at addressing some key questions in this domain: (a) is this (new) regulation relevant for me? (b) what set of requirements does this law impose?, and (c) what is the regulatory intent of a law? The system is currently undergoing user trials within our organization.
A Flexible and Easy-to-use Semantic Role Labeling Framework for Different Languages
Authors: Quynh Ngoc Thi Do, Artuur Leeuwenberg, Geert Heyman, Marie-Francine Moens
Date: 2018-08-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Feature Engineering, Semantic Role Labeling

This paper presents a flexible and open source framework for deep semantic role labeling. We aim at facilitating easy exploration of model structures for multiple languages with different characteristics. It provides flexibility in its model construction in terms of word representation, sequence representation, output modeling, and inference styles and comes with clear output visualization. The framework is available under the Apache 2.0 license.
Boosting for Efficient Model Selection for Syntactic Parsing
Authors: Rachel Bawden, Beno{\^\i}t Crabb{\'e}
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Constituency Parsing, model, Model Selection

We present an efficient model selection method using boosting for transition-based constituency parsing. It is designed for exploring a high-dimensional search space, defined by a large set of feature templates, as for example is typically the case when parsing morphologically rich languages. Our method removes the need to manually define heuristic constraints, which are often imposed in current state-of-the-art selection methods. Our experiments for French show that the method is more efficient and is also capable of producing compact, state-of-the-art models.
A Universal Framework for Inductive Transfer Parsing across Multi-typed Treebanks
Authors: Jiang Guo, Wanxiang Che, Haifeng Wang, Ting Liu
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Dependency Parsing, Information Retrieval, Multi-Task Learning

Various treebanks have been released for dependency parsing. Despite that treebanks may belong to different languages or have different annotation schemes, they contain common syntactic knowledge that is potential to benefit each other. This paper presents a universal framework for transfer parsing across multi-typed treebanks with deep multi-task learning. We consider two kinds of treebanks as source: the multilingual universal treebanks and the monolingual heterogeneous treebanks. Knowledge across the source and target treebanks are effectively transferred through multi-level parameter sharing. Experiments on several benchmark datasets in various languages demonstrate that our approach can make effective use of arbitrary source treebanks to improve target parsing models.
Grammar induction from (lots of) words alone
Authors: John K Pate, Mark Johnson
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Language Acquisition, POS, TAG, Topic Models

Grammar induction is the task of learning syntactic structure in a setting where that structure is hidden. Grammar induction from words alone is interesting because it is similiar to the problem that a child learning a language faces. Previous work has typically assumed richer but cognitively implausible input, such as POS tag annotated data, which makes that work less relevant to human language acquisition. We show that grammar induction from words alone is in fact feasible when the model is provided with sufficient training data, and present two new streaming or mini-batch algorithms for PCFG inference that can learn from millions of words of training data. We compare the performance of these algorithms to a batch algorithm that learns from less data. The minibatch algorithms outperform the batch algorithm, showing that cheap inference with more data is better than intensive inference with less data. Additionally, we show that the harmonic initialiser, which previous work identified as essential when learning from small POS-tag annotated corpora (Klein and Manning, 2004), is not superior to a uniform initialisation.
A Redundancy-Aware Sentence Regression Framework for Extractive Summarization
Authors: Pengjie Ren, Furu Wei, Zhumin Chen, Jun Ma, Ming Zhou
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Document Summarization, Extractive Summarization, Multi-Document Summarization, regression, Sentence

Existing sentence regression methods for extractive summarization usually model sentence importance and redundancy in two separate processes. They first evaluate the importance f(s) of each sentence s and then select sentences to generate a summary based on both the importance scores and redundancy among sentences. In this paper, we propose to model importance and redundancy simultaneously by directly evaluating the relative importance f(s|S) of a sentence s given a set of selected sentences S. Specifically, we present a new framework to conduct regression with respect to the relative gain of s given S calculated by the ROUGE metric. Besides the single sentence features, additional features derived from the sentence relations are incorporated. Experiments on the DUC 2001, 2002 and 2004 multi-document summarization datasets show that the proposed method outperforms state-of-the-art extractive summarization approaches.
Generating Video Description using Sequence-to-sequence Model with Temporal Attention
Authors: Natsuda Laokulrat, Sang Phan, Noriki Nishida, Raphael Shu, Yo Ehara, Naoaki Okazaki, Yusuke Miyao, Hideki Nakayama
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Caption Generation, Sentence, Video Classification, Video Description

Automatic video description generation has recently been getting attention after rapid advancement in image caption generation. Automatically generating description for a video is more challenging than for an image due to its temporal dynamics of frames. Most of the work relied on Recurrent Neural Network (RNN) and recently attentional mechanisms have also been applied to make the model learn to focus on some frames of the video while generating each word in a describing sentence. In this paper, we focus on a sequence-to-sequence approach with temporal attention mechanism. We analyze and compare the results from different attention model configuration. By applying the temporal attention mechanism to the system, we can achieve a METEOR score of 0.310 on Microsoft Video Description dataset, which outperformed the state-of-the-art system so far.
CATENA: CAusal and TEmporal relation extraction from NAtural language texts
Authors: Paramita Mirza, Sara Tonelli
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: General Classification, Question Answering, Relation, Relation Classification, Relation Extraction, Temporal Information Extraction, Temporal Relation Extraction

We present CATENA, a sieve-based system to perform temporal and causal relation extraction and classification from English texts, exploiting the interaction between the temporal and the causal model. We evaluate the performance of each sieve, showing that the rule-based, the machine-learned and the reasoning components all contribute to achieving state-of-the-art performance on TempEval-3 and TimeBank-Dense data. Although causal relations are much sparser than temporal ones, the architecture and the selected features are mostly suitable to serve both tasks. The effects of the interaction between the temporal and the causal components, although limited, yield promising results and confirm the tight connection between the temporal and the causal dimension of texts.
Forecasting Word Model: Twitter-based Influenza Surveillance and Prediction
Authors: Hayate Iso, Shoko Wakamiya, Eiji Aramaki
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Future prediction, Prediction

Because of the increasing popularity of social media, much information has been shared on the internet, enabling social media users to understand various real world events. Particularly, social media-based infectious disease surveillance has attracted increasing attention. In this work, we specifically examine influenza: a common topic of communication on social media. The fundamental theory of this work is that several words, such as symptom words (fever, headache, etc.), appear in advance of flu epidemic occurrence. Consequently, past word occurrence can contribute to estimation of the number of current patients. To employ such forecasting words, one can first estimate the optimal time lag for each word based on their cross correlation. Then one can build a linear model consisting of word frequencies at different time points for nowcasting and for forecasting influenza epidemics. Experimentally obtained results (using 7.7 million tweets of August 2012 {--} January 2016), the proposed model achieved the best nowcasting performance to date (correlation ratio 0.93) and practically sufficient forecasting performance (correlation ratio 0.91 in 1-week future prediction, and correlation ratio 0.77 in 3-weeks future prediction). This report is the first of the relevant literature to describe a model enabling prediction of future epidemics using Twitter.
Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity
Authors: Nils Reimers, Philip Beyer, Iryna Gurevych
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Question Answering, Semantic Similarity, Semantic Textual Similarity, STS, Text Summarization

Semantic Textual Similarity (STS) is a foundational NLP task and can be used in a wide range of tasks. To determine the STS of two texts, hundreds of different STS systems exist, however, for an NLP system designer, it is hard to decide which system is the best one. To answer this question, an intrinsic evaluation of the STS systems is conducted by comparing the output of the system to human judgments on semantic similarity. The comparison is usually done using Pearson correlation. In this work, we show that relying on intrinsic evaluations with Pearson correlation can be misleading. In three common STS based tasks we could observe that the Pearson correlation was especially ill-suited to detect the best STS system for the task and other evaluation measures were much better suited. In this work we define how the validity of an intrinsic evaluation can be assessed and compare different intrinsic evaluation methods. Understanding of the properties of the targeted task is crucial and we propose a framework for conducting the intrinsic evaluation which takes the properties of the targeted task into account.
Expanding wordnets to new languages with multilingual sense disambiguation
Authors: Mihael Arcan, John Philip McCrae, Paul Buitelaar
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Information Retrieval, Machine Translation, Sentiment Analysis, Translation, Word Sense Disambiguation

Princeton WordNet is one of the most important resources for natural language processing, but is only available for English. While it has been translated using the expand approach to many other languages, this is an expensive manual process. Therefore it would be beneficial to have a high-quality automatic translation approach that would support NLP techniques, which rely on WordNet in new languages. The translation of wordnets is fundamentally complex because of the need to translate all senses of a word including low frequency senses, which is very challenging for current machine translation approaches. For this reason we leverage existing translations of WordNet in other languages to identify contextual information for wordnet senses from a large set of generic parallel corpora. We evaluate our approach using 10 translated wordnets for European languages. Our experiment shows a significant improvement over translation without any contextual information. Furthermore, we evaluate how the choice of pivot languages affects performance of multilingual word sense disambiguation.
Zero-resource Dependency Parsing: Boosting Delexicalized Cross-lingual Transfer with Linguistic Knowledge
Authors: Lauriane Aufrant, Guillaume Wisniewski, Fran{\c{c}}ois Yvon
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Active Learning, Cross-Lingual Transfer, Dependency Parsing, Machine Translation, POS

This paper studies cross-lingual transfer for dependency parsing, focusing on very low-resource settings where delexicalized transfer is the only fully automatic option. We show how to boost parsing performance by rewriting the source sentences so as to better match the linguistic regularities of the target language. We contrast a data-driven approach with an approach relying on linguistically motivated rules automatically extracted from the World Atlas of Language Structures. Our findings are backed up by experiments involving 40 languages. They show that both approaches greatly outperform the baseline, the knowledge-driven method yielding the best accuracies, with average improvements of +2.9 UAS, and up to +90 UAS (absolute) on some frequent PoS configurations.
Deceptive Opinion Spam Detection Using Neural Network
Authors: Yafeng Ren, Yue Zhang
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Sentence, Spam detection, Word Embeddings

Deceptive opinion spam detection has attracted significant attention from both business and research communities. Existing approaches are based on manual discrete features, which can capture linguistic and psychological cues. However, such features fail to encode the semantic meaning of a document from the discourse perspective, which limits the performance. In this paper, we empirically explore a neural network model to learn document-level representation for detecting deceptive opinion spam. In particular, given a document, the model learns sentence representations with a convolutional neural network, which are combined using a gated recurrent neural network with attention mechanism to model discourse information and yield a document vector. Finally, the document representation is used directly as features to identify deceptive opinion spam. Experimental results on three domains (Hotel, Restaurant, and Doctor) show that our proposed method outperforms state-of-the-art methods.
Integrating Topic Modeling with Word Embeddings by Mixtures of vMFs
Authors: Xi-Ming Li, Jinjin Chi, Changchun Li, Jihong Ouyang, Bo Fu
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Topic Models, Word Embeddings

Gaussian LDA integrates topic modeling with word embeddings by replacing discrete topic distribution over word types with multivariate Gaussian distribution on the embedding space. This can take semantic information of words into account. However, the Euclidean similarity used in Gaussian topics is not an optimal semantic measure for word embeddings. Acknowledgedly, the cosine similarity better describes the semantic relatedness between word embeddings. To employ the cosine measure and capture complex topic structure, we use von Mises-Fisher (vMF) mixture models to represent topics, and then develop a novel mix-vMF topic model (MvTM). Using public pre-trained word embeddings, we evaluate MvTM on three real-world data sets. Experimental results show that our model can discover more coherent topics than the state-of-the-art baseline models, and achieve competitive classification performance.
Bayesian Language Model based on Mixture of Segmental Contexts for Spontaneous Utterances with Unexpected Words
Authors: Ryu Takeda, Kazunori Komatani
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, Speech Recognition

This paper describes a Bayesian language model for predicting spontaneous utterances. People sometimes say unexpected words, such as fillers or hesitations, that cause the miss-prediction of words in normal N-gram models. Our proposed model considers mixtures of possible segmental contexts, that is, a kind of context-word selection. It can reduce negative effects caused by unexpected words because it represents conditional occurrence probabilities of a word as weighted mixtures of possible segmental contexts. The tuning of mixture weights is the key issue in this approach as the segment patterns becomes numerous, thus we resolve it by using Bayesian model. The generative process is achieved by combining the stick-breaking process and the process used in the variable order Pitman-Yor language model. Experimental evaluations revealed that our model outperformed contiguous N-gram models in terms of perplexity for noisy text including hesitations.
Label Embedding for Zero-shot Fine-grained Named Entity Typing
Authors: Yukun Ma, Erik Cambria, Sa Gao
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Entity Linking, Entity Typing, Named Entity Recognition (NER), Question Answering, Sentiment Analysis, Zero-Shot Learning

Named entity typing is the task of detecting the types of a named entity in context. For instance, given {``}Eric is giving a presentation{''}, our goal is to infer that {`}Eric{'} is a speaker or a presenter and a person. Existing approaches to named entity typing cannot work with a growing type set and fails to recognize entity mentions of unseen types. In this paper, we present a label embedding method that incorporates prototypical and hierarchical information to learn pre-trained label embeddings. In addition, we adapt a zero-shot learning framework that can predict both seen and previously unseen entity types. We perform evaluation on three benchmark datasets with two settings: 1) few-shots recognition where all types are covered by the training set; and 2) zero-shot recognition where fine-grained types are assumed absent from training set. Results show that prior knowledge encoded using our label embedding methods can significantly boost the performance of classification for both cases.
The Role of Context in Neural Morphological Disambiguation
Authors: Qinlan Shen, Daniel Clothiaux, Emily Tagtow, Patrick Littell, Chris Dyer
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Morphological Disambiguation

Languages with rich morphology often introduce sparsity in language processing tasks. While morphological analyzers can reduce this sparsity by providing morpheme-level analyses for words, they will often introduce ambiguity by returning multiple analyses for the same surface form. The problem of disambiguating between these morphological parses is further complicated by the fact that a correct parse for a word is not only be dependent on the surface form but also on other words in its context. In this paper, we present a language-agnostic approach to morphological disambiguation. We address the problem of using context in morphological disambiguation by presenting several LSTM-based neural architectures that encode long-range surface-level and analysis-level contextual dependencies. We applied our approach to Turkish, Russian, and Arabic to compare effectiveness across languages, matching state-of-the-art results in two of the three languages. Our results also demonstrate that while context plays a role in learning how to disambiguate, the type and amount of context needed varies between languages.
Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features
Authors: Xu Sun
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Low-Rank Matrix Completion, Matrix Completion

Existing asynchronous parallel learning methods are only for the sparse feature models, and they face new challenges for the dense feature models like neural networks (e.g., LSTM, RNN). The problem for dense features is that asynchronous parallel learning brings gradient errors derived from overwrite actions. We show that gradient errors are very common and inevitable. Nevertheless, our theoretical analysis shows that the learning process with gradient errors can still be convergent towards the optimum of objective functions for many practical applications. Thus, we propose a simple method \textit{AsynGrad} for asynchronous parallel learning with gradient error. Base on various dense feature models (LSTM, dense-CRF) and various NLP tasks, experiments show that \textit{AsynGrad} achieves substantial improvement on training speed, and without any loss on accuracy.
Exploring Text Links for Coherent Multi-Document Summarization
Authors: Xun Wang, Masaaki Nishino, Tsutomu Hirao, Katsuhito Sudoh, Masaaki Nagata
Date: 2016-12-01 00:00:00
Proceeding: COLING 2016 12
Tasks: Document Summarization, Informativeness, Multi-Document Summarization, Text Generation

Summarization aims to represent source documents by a shortened passage. Existing methods focus on the extraction of key information, but often neglect coherence. Hence the generated summaries suffer from a lack of readability. To address this problem, we have developed a graph-based method by exploring the links between text to produce coherent summaries. Our approach involves finding a sequence of sentences that best represent the key information in a coherent way. In contrast to the previous methods that focus only on salience, the proposed method addresses both coherence and informativeness based on textual linkages. We conduct experiments on the DUC2004 summarization task data set. A performance comparison reveals that the summaries generated by the proposed system achieve comparable results in terms of the ROUGE metric, and show improvements in readability by human evaluation.