pwc-archive/papers-with-abstracts huggingface dataset:
Context-aware Interactive Attention for Multi-modal Sentiment and Emotion Analysis
Authors: Dushyant Singh Chauhan, Md. Shad Akhtar, Asif Ekbal, Pushpak Bhattacharyya
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Emotion Recognition

In recent times, multi-modal analysis has been an emerging and highly sought-after field at the intersection of natural language processing, computer vision, and speech processing. The prime objective of such studies is to leverage the diversified information, (e.g., textual, acoustic and visual), for learning a model. The effective interaction among these modalities often leads to a better system in terms of performance. In this paper, we introduce a recurrent neural network based approach for the multi-modal sentiment and emotion analysis. The proposed model learns the inter-modal interaction among the participating modalities through an auto-encoder mechanism. We employ a context-aware attention module to exploit the correspondence among the neighboring utterances. We evaluate our proposed approach for five standard multi-modal affect analysis datasets. Experimental results suggest the efficacy of the proposed model for both sentiment and emotion analysis over various existing state-of-the-art systems.
Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree
Authors: Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, Xudong Liu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Representation Learning, Sentence, Sentiment Analysis, Sentiment Classification

We propose a method based on neural networks to identify the sentiment polarity of opinion words expressed on a specific aspect of a sentence. Although a large majority of works typically focus on leveraging the expressive power of neural networks in handling this task, we explore the possibility of integrating dependency trees with neural networks for representation learning. To this end, we present a convolution over a dependency tree (CDT) model which exploits a Bi-directional Long Short Term Memory (Bi-LSTM) to learn representations for features of a sentence, and further enhance the embeddings with a graph convolutional network (GCN) which operates directly on the dependency tree of the sentence. Our approach propagates both contextual and dependency information from opinion words to aspect words, offering discriminative properties for supervision. Experimental results ranks our approach as the new state-of-the-art in aspect-based sentiment classification.
Understanding Data Augmentation in Neural Machine Translation: Two Perspectives towards Generalization
Authors: Guanlin Li, Lemao Liu, Guoping Huang, Conghui Zhu, Tiejun Zhao
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Data Augmentation, Machine Translation, Translation, Vocal Bursts Valence Prediction

Many Data Augmentation (DA) methods have been proposed for neural machine translation. Existing works measure the superiority of DA methods in terms of their performance on a specific test set, but we find that some DA methods do not exhibit consistent improvements across translation tasks. Based on the observation, this paper makes an initial attempt to answer a fundamental question: what benefits, which are consistent across different methods and tasks, does DA in general obtain? Inspired by recent theoretic advances in deep learning, the paper understands DA from two perspectives towards the generalization ability of a model: input sensitivity and prediction margin, which are defined independent of specific test set thereby may lead to findings with relatively low variance. Extensive experiments show that relatively consistent benefits across five DA methods and four translation tasks are achieved regarding both perspectives.
Multilingual Grammar Induction with Continuous Language Identification
Authors: Wenjuan Han, Ge Wang, Yong Jiang, Kewei Tu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Language Identification

The key to multilingual grammar induction is to couple grammar parameters of different languages together by exploiting the similarity between languages. Previous work relies on linguistic phylogenetic knowledge to specify similarity between languages. In this work, we propose a novel universal grammar induction approach that represents language identities with continuous vectors and employs a neural network to predict grammar parameters based on the representation. Without any prior linguistic phylogenetic knowledge, we automatically capture similarity between languages with the vector representations and softly tie the grammar parameters of different languages. In our experiments, we apply our approach to 15 languages across 8 language families and subfamilies in the Universal Dependency Treebank dataset, and we observe substantial performance gain on average over monolingual and multilingual baselines.
Event Detection with Multi-Order Graph Convolution and Aggregated Attention
Authors: Haoran Yan, Xiaolong Jin, Xiangbin Meng, Jiafeng Guo, Xue-Qi Cheng
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Event Detection, Sentence

Syntactic relations are broadly used in many NLP tasks. For event detection, syntactic relation representations based on dependency tree can better capture the interrelations between candidate trigger words and related entities than sentence representations. But, existing studies only use first-order syntactic relations (i.e., the arcs) in dependency trees to identify trigger words. For this reason, this paper proposes a new method for event detection, which uses a dependency tree based graph convolution network with aggregative attention to explicitly model and aggregate multi-order syntactic representations in sentences. Experimental comparison with state-of-the-art baselines shows the superiority of the proposed method.
Coverage of Information Extraction from Sentences and Paragraphs
Authors: Simon Razniewski, Nitisha Jain, Paramita Mirza, Gerhard Weikum
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Implicatures, Negation

Scalar implicatures are language features that imply the negation of stronger statements, e.g., {``}She was married twice{''} typically implicates that she was not married thrice. In this paper we discuss the importance of scalar implicatures in the context of textual information extraction. We investigate how textual features can be used to predict whether a given text segment mentions all objects standing in a certain relationship with a certain subject. Preliminary results on Wikipedia indicate that this prediction is feasible, and yields informative assessments.
HMEAE: Hierarchical Modular Event Argument Extraction
Authors: Xiaozhi Wang, Ziqi Wang, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Maosong Sun, Jie zhou, Xiang Ren
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Event Argument Extraction, Event Extraction, Inductive Bias

Existing event extraction methods classify each argument role independently, ignoring the conceptual correlations between different argument roles. In this paper, we propose a Hierarchical Modular Event Argument Extraction (HMEAE) model, to provide effective inductive bias from the concept hierarchy of event argument roles. Specifically, we design a neural module network for each basic unit of the concept hierarchy, and then hierarchically compose relevant unit modules with logical operations into a role-oriented modular network to classify a specific argument role. As many argument roles share the same high-level unit module, their correlation can be utilized to extract specific event arguments better. Experiments on real-world datasets show that HMEAE can effectively leverage useful knowledge from the concept hierarchy and significantly outperform the state-of-the-art baselines. The source code can be obtained from https://github.com/thunlp/HMEAE.
Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains
Authors: Wei Shi, Vera Demberg
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Discourse Parsing, General Classification, Implicit Discourse Relation Classification, Relation, Relation Classification, Sentence

Implicit discourse relation classification is one of the most difficult tasks in discourse parsing. Previous studies have generally focused on extracting better representations of the relational arguments. In order to solve the task, it is however additionally necessary to capture what events are expected to cause or follow each other. Current discourse relation classifiers fall short in this respect. We here show that this shortcoming can be effectively addressed by using the bidirectional encoder representation from transformers (BERT) proposed by Devlin et al. (2019), which were trained on a next-sentence prediction task, and thus encode a representation of likely next sentences. The BERT-based model outperforms the current state of the art in 11-way classification by 8{\%} points on the standard PDTB dataset. Our experiments also demonstrate that the model can be successfully ported to other domains: on the BioDRB dataset, the model outperforms the state of the art system around 15{\%} points.
Split or Merge: Which is Better for Unsupervised RST Parsing?
Authors: Naoki Kobayashi, Tsutomu Hirao, Kengo Nakamura, Hidetaka Kamigaito, Manabu Okumura, Masaaki Nagata
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Rhetorical Structure Theory (RST) parsing is crucial for many downstream NLP tasks that require a discourse structure for a text. Most of the previous RST parsers have been based on supervised learning approaches. That is, they require an annotated corpus of sufficient size and quality, and heavily rely on the language and domain dependent corpus. In this paper, we present two language-independent unsupervised RST parsing methods based on dynamic programming. The first one builds the optimal tree in terms of a dissimilarity score function that is defined for splitting a text span into smaller ones. The second builds the optimal tree in terms of a similarity score function that is defined for merging two adjacent spans into a large one. Experimental results on English and German RST treebanks showed that our parser based on span merging achieved the best score, around 0.8 F$_1$ score, which is close to the scores of the previous supervised parsers.
Event Causality Recognition Exploiting Multiple Annotators' Judgments and Background Knowledge
Authors: Kazuma Kadowaki, Ryu Iida, Kentaro Torisawa, Jong-Hoon Oh, Julien Kloetzer
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Articles

We propose new BERT-based methods for recognizing event causality such as {``}smoke cigarettes{''} {--}{\textgreater} {``}die of lung cancer{''} written in web texts. In our methods, we grasp each annotator{'}s policy by training multiple classifiers, each of which predicts the labels given by a single annotator, and combine the resulting classifiers{'} outputs to predict the final labels determined by majority vote. Furthermore, we investigate the effect of supplying background knowledge to our classifiers. Since BERT models are pre-trained with a large corpus, some sort of background knowledge for event causality may be learned during pre-training. Our experiments with a Japanese dataset suggest that this is actually the case: Performance improved when we pre-trained the BERT models with web texts containing a large number of event causalities instead of Wikipedia articles or randomly sampled web texts. However, this effect was limited. Therefore, we further improved performance by simply adding texts related to an input causality candidate as background knowledge to the input of the BERT models. We believe these findings indicate a promising future research direction.
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons
Authors: Ji Xin, Jimmy Lin, Yao-Liang Yu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Memory neurons of long short-term memory (LSTM) networks encode and process information in powerful yet mysterious ways. While there has been work to analyze their behavior in carrying low-level information such as linguistic properties, how they directly contribute to label prediction remains unclear. We find inspiration from biologists and study the affinity between individual neurons and labels, propose a novel metric to quantify the sensitivity of neurons to each label, and conduct experiments to show the validity of our proposed metric. We discover that some neurons are trained to specialize on a subset of labels, and while dropping an arbitrary neuron has little effect on the overall accuracy of the model, dropping label-specialized neurons predictably and significantly degrades prediction accuracy on the associated label. We further examine the consistency of neuron-label affinity across different models. These observations provide insight into the inner mechanisms of LSTMs.
Text Genre and Training Data Size in Human-like Parsing
Authors: John Hale, Adhiguna Kuncoro, Keith Hall, Chris Dyer, Jonathan Brennan
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Domain-specific training typically makes NLP systems work better. We show that this extends to cognitive modeling as well by relating the states of a neural phrase-structure parser to electrophysiological measures from human participants. These measures were recorded as participants listened to a spoken recitation of the same literary text that was supplied as input to the neural parser. Given more training data, the system derives a better cognitive model {---} but only when the training examples come from the same textual genre. This finding is consistent with the idea that humans adapt syntactic expectations to particular genres during language comprehension (Kaan and Chun, 2018; Branigan and Pickering, 2017).
Revisiting the Evaluation of Theory of Mind through Question Answering
Authors: Matthew Le, Y-Lan Boureau, Maximilian Nickel
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Question Answering

Theory of mind, i.e., the ability to reason about intents and beliefs of agents is an important task in artificial intelligence and central to resolving ambiguous references in natural language dialogue. In this work, we revisit the evaluation of theory of mind through question answering. We show that current evaluation methods are flawed and that existing benchmark tasks can be solved without theory of mind due to dataset biases. Based on prior work, we propose an improved evaluation protocol and dataset in which we explicitly control for data regularities via a careful examination of the answer space. We show that state-of-the-art methods which are successful on existing benchmarks fail to solve theory-of-mind tasks in our proposed approach.
MICRON: Multigranular Interaction for Contextualizing RepresentatiON in Non-factoid Question Answering
Authors: Hojae Han, Seungtaek Choi, Haeju Park, Seung-won Hwang
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Information Retrieval, Question Answering, Retrieval

This paper studies the problem of non-factoid question answering, where the answer may span over multiple sentences. Existing solutions can be categorized into representation- and interaction-focused approaches. We combine their complementary strength, by a hybrid approach allowing multi-granular interactions, but represented at word level, enabling an easy integration with strong word-level signals. Specifically, we propose MICRON: Multigranular Interaction for Contextualizing RepresentatiON, a novel approach which derives contextualized uni-gram representation from n-grams. Our contributions are as follows: First, we enable multi-granular matches between question and answer $n$-grams. Second, by contextualizing word representation with surrounding n-grams, MICRON can naturally utilize word-based signals for query term weighting, known to be effective in information retrieval. We validate MICRON in two public non-factoid question answering datasets: WikiPassageQA and InsuranceQA, showing our model achieves the state of the art among baselines with reported performances on both datasets.
Machine Reading Comprehension Using Structural Knowledge Graph-aware Network
Authors: Delai Qiu, Yuanzhe Zhang, Xinwei Feng, Xiangwen Liao, Wenbin Jiang, Yajuan Lyu, Kang Liu, Jun Zhao
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Knowledge Graphs, Machine Reading Comprehension, Reading Comprehension

Leveraging external knowledge is an emerging trend in machine comprehension task. Previous work usually utilizes knowledge graphs such as ConceptNet as external knowledge, and extracts triples from them to enhance the initial representation of the machine comprehension context. However, such method cannot capture the structural information in the knowledge graph. To this end, we propose a Structural Knowledge Graph-aware Network(SKG) model, constructing sub-graphs for entities in the machine comprehension context. Our method dynamically updates the representation of the knowledge according to the structural information of the constructed sub-graph. Experiments show that SKG achieves state-of-the-art performance on the ReCoRD dataset.
Improving Answer Selection and Answer Triggering using Hard Negatives
Authors: Sawan Kumar, Shweta Garg, Kartik Mehta, Nikhil Rasiwasia
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Answer Selection

In this paper, we establish the effectiveness of using hard negatives, coupled with a siamese network and a suitable loss function, for the tasks of answer selection and answer triggering. We show that the choice of sampling strategy is key for achieving improved performance on these tasks. Evaluating on recent answer selection datasets - InsuranceQA, SelQA, and an internal QA dataset, we show that using hard negatives with relatively simple model architectures (bag of words and LSTM-CNN) drives significant performance gains. On InsuranceQA, this strategy alone improves over previously reported results by a minimum of 1.6 points in P@1. Using hard negatives with a Transformer encoder provides a further improvement of 2.3 points. Further, we propose to use quadruplet loss for answer triggering, with the aim of producing globally meaningful similarity scores. We show that quadruplet loss function coupled with the selection of hard negatives enables bag-of-words models to improve F1 score by 2.3 points over previous baselines, on SelQA answer triggering dataset. Our results provide key insights into answer selection and answer triggering tasks.
Can You Unpack That? Learning to Rewrite Questions-in-Context
Authors: Ahmed Elgohary, Denis Peskov, Jordan Boyd-Graber
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Question Answering

Question answering is an AI-complete problem, but existing datasets lack key elements of language understanding such as coreference and ellipsis resolution. We consider sequential question answering: multiple questions are asked one-by-one in a conversation between a questioner and an answerer. Answering these questions is only possible through understanding the conversation history. We introduce the task of question-in-context rewriting: given the context of a conversation{'}s history, rewrite a context-dependent into a self-contained question with the same answer. We construct, CANARD, a dataset of 40,527 questions based on QuAC (Choi et al., 2018) and train Seq2Seq models for incorporating context into standalone questions.
Query-focused Sentence Compression in Linear Time
Authors: H, Abram ler, Brendan O{'}Connor
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: GPU, Sentence, Sentence Compression

Search applications often display shortened sentences which must contain certain query terms and must fit within the space constraints of a user interface. This work introduces a new transition-based sentence compression technique developed for such settings. Our query-focused method constructs length and lexically constrained compressions in linear time, by growing a subgraph in the dependency parse of a sentence. This theoretically efficient approach achieves an 11x empirical speedup over baseline ILP methods, while better reconstructing gold constrained shortenings. Such speedups help query-focused applications, because users are measurably hindered by interface lags. Additionally, our technique does not require an ILP solver or a GPU.
Abstract Text Summarization: A Low Resource Challenge
Authors: Shantipriya Parida, Petr Motlicek
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Data Augmentation, Text Summarization

Text summarization is considered as a challenging task in the NLP community. The availability of datasets for the task of multilingual text summarization is rare, and such datasets are difficult to construct. In this work, we build an abstract text summarizer for the German language text using the state-of-the-art {``}Transformer{''} model. We propose an iterative data augmentation approach which uses synthetic data along with the real summarization data for the German language. To generate synthetic data, the Common Crawl (German) dataset is exploited, which covers different domains. The synthetic data is effective for the low resource condition and is particularly helpful for our multilingual scenario where availability of summarizing data is still a challenging issue. The data are also useful in deep learning scenarios where the neural models require a large amount of training data for utilization of its capacity. The obtained summarization performance is measured in terms of ROUGE and BLEU score. We achieve an absolute improvement of +1.5 and +16.0 in ROUGE1 F1 (R1{\_}F1) on the development and test sets, respectively, compared to the system which does not rely on data augmentation.
Generating Modern Poetry Automatically in Finnish
Authors: Mika H{\"a}m{\"a}l{\"a}inen, Khalid Alnajjar
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

We present a novel approach for generating poetry automatically for the morphologically rich Finnish language by using a genetic algorithm. The approach improves the state of the art of the previous Finnish poem generators by introducing a higher degree of freedom in terms of structural creativity. Our approach is evaluated and described within the paradigm of computational creativity, where the fitness functions of the genetic algorithm are assimilated with the notion of aesthetics. The output is considered to be a poem 81.5{\%} of the time by human evaluators.
SUM-QE: a BERT-based Summary Quality Estimation Model
Authors: Stratos Xenouleas, Prodromos Malakasiotis, Marianna Apidianaki, Ion Androutsopoulos
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

We propose SUM-QE, a novel Quality Estimation model for summarization based on BERT. The model addresses linguistic quality aspects that are only indirectly captured by content-based approaches to summary evaluation, without involving comparison with human references. SUM-QE achieves very high correlations with human ratings, outperforming simpler models addressing these linguistic aspects. Predictions of the SUM-QE model can be used for system development, and to inform users of the quality of automatically produced summaries and other types of generated text.