pwc-archive/papers-with-abstracts huggingface dataset:
Synchronously Generating Two Languages with Interactive Decoding
Authors: Yining Wang, Jiajun Zhang, Long Zhou, Yuchen Liu, Cheng-qing Zong
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, NMT, Translation, Vocal Bursts Valence Prediction

In this paper, we introduce a novel interactive approach to translate a source language into two different languages simultaneously and interactively. Specifically, the generation of one language relies on not only previously generated outputs by itself, but also the outputs predicted in the other language. Experimental results on IWSLT and WMT datasets demonstrate that our method can obtain significant improvements over both conventional Neural Machine Translation (NMT) model and multilingual NMT model.
``Going on a vacation'' takes longer than ``Going for a walk'': A Study of Temporal Commonsense Understanding
Authors: Ben Zhou, Daniel Khashabi, Qiang Ning, Dan Roth
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Understanding time is crucial for understanding events expressed in natural language. Because people rarely say the obvious, it is often necessary to have commonsense knowledge about various temporal aspects of events, such as duration, frequency, and temporal order. However, this important problem has so far received limited attention. This paper systematically studies this temporal commonsense problem. Specifically, we define five classes of temporal commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that serves as a test set for this task. We find that the best current methods used on MCTACO are still far behind human performance, by about 20{\%}, and discuss several directions for improvement. We hope that the new dataset and our study here can foster more future research on this topic.
Recognizing Conflict Opinions in Aspect-level Sentiment Classification with Dual Attention Networks
Authors: Xingwei Tan, Yi Cai, Changxi Zhu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Sentiment Analysis, Sentiment Classification

Aspect-level sentiment classification, which is a fine-grained sentiment analysis task, has received lots of attention these years. There is a phenomenon that people express both positive and negative sentiments towards an aspect at the same time. Such opinions with conflicting sentiments, however, are ignored by existing studies, which design models based on the absence of them. We argue that the exclusion of conflict opinions is problematic, for the reason that it represents an important style of human thinking {--} dialectic thinking. If a real-world sentiment classification system ignores the existence of conflict opinions when it is designed, it will incorrectly mixed conflict opinions into other sentiment polarity categories in action. Existing models have problems when recognizing conflicting opinions, such as data sparsity. In this paper, we propose a multi-label classification model with dual attention mechanism to address these problems.
Investigating Dynamic Routing in Tree-Structured LSTM for Sentiment Analysis
Authors: Jin Wang, Liang-Chih Yu, K. Robert Lai, Xue-jie Zhang
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Sentence, Sentiment Analysis, Sentiment Classification

Deep neural network models such as long short-term memory (LSTM) and tree-LSTM have been proven to be effective for sentiment analysis. However, sequential LSTM is a bias model wherein the words in the tail of a sentence are more heavily emphasized than those in the header for building sentence representations. Even tree-LSTM, with useful structural information, could not avoid the bias problem because the root node will be dominant and the nodes in the bottom of the parse tree will be less emphasized even though they may contain salient information. To overcome the bias problem, this study proposes a capsule tree-LSTM model, introducing a dynamic routing algorithm as an aggregation layer to build sentence representation by assigning different weights to nodes according to their contributions to prediction. Experiments on Stanford Sentiment Treebank (SST) for sentiment classification and EmoBank for regression show that the proposed method improved the performance of tree-LSTM and other neural network models. In addition, the deeper the tree structure, the bigger the improvement.
A Label Informative Wide \& Deep Classifier for Patents and Papers
Authors: Muyao Niu, Jie Cai
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Patent classification

In this paper, we provide a simple and effective baseline for classifying both patents and papers to the well-established Cooperative Patent Classification (CPC). We propose a label-informative classifier based on the Wide {\&} Deep structure, where the Wide part encodes string-level similarities between texts and labels, and the Deep part captures semantic-level similarities via non-linear transformations. Our model trains on millions of patents, and transfers to papers by developing distant-supervised training set and domain-specific features. Extensive experiments show that our model achieves comparable performance to the state-of-the-art model used in industry on both patents and papers. The output of this work should facilitate the searching, granting and filing of innovative ideas for patent examiners, attorneys and researchers.
Visual Detection with Context for Document Layout Analysis
Authors: Carlos Soto, Shinjae Yoo
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Articles, Document Layout Analysis, Literature Mining, object-detection, Object Detection

We present 1) a work in progress method to visually segment key regions of scientific articles using an object detection technique augmented with contextual features, and 2) a novel dataset of region-labeled articles. A continuing challenge in scientific literature mining is the difficulty of consistently extracting high-quality text from formatted PDFs. To address this, we adapt the object-detection technique Faster R-CNN for document layout detection, incorporating contextual information that leverages the inherently localized nature of article contents to improve the region detection performance. Due to the limited availability of high-quality region-labels for scientific articles, we also contribute a novel dataset of region annotations, the first version of which covers 9 region classes and 822 article pages. Initial experimental results demonstrate a 23.9{\%} absolute improvement in mean average precision over the baseline model by incorporating contextual features, and a processing speed 14x faster than a text-based technique. Ongoing work on further improvements is also discussed.
Neural Topic Model with Reinforcement Learning
Authors: Lin Gui, Jia Leng, Gabriele Pergola, Yu Zhou, Ruifeng Xu, Yulan He
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: model, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Topic Models, Variational Inference

In recent years, advances in neural variational inference have achieved many successes in text processing. Examples include neural topic models which are typically built upon variational autoencoder (VAE) with an objective of minimising the error of reconstructing original documents based on the learned latent topic vectors. However, minimising reconstruction errors does not necessarily lead to high quality topics. In this paper, we borrow the idea of reinforcement learning and incorporate topic coherence measures as reward signals to guide the learning of a VAE-based topic model. Furthermore, our proposed model is able to automatically separating background words dynamically from topic words, thus eliminating the pre-processing step of filtering infrequent and/or top frequent words, typically required for learning traditional topic models. Experimental results on the 20 Newsgroups and the NIPS datasets show superior performance both on perplexity and topic coherence measure compared to state-of-the-art neural topic models.
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval
Authors: Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, Jimmy Lin
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Articles, Retrieval, Sentence

This paper applies BERT to ad hoc document retrieval on news articles, which requires addressing two challenges: relevance judgments in existing test collections are typically provided only at the document level, and documents often exceed the length that BERT was designed to handle. Our solution is to aggregate sentence-level evidence to rank documents. Furthermore, we are able to leverage passage-level relevance judgments fortuitously available in other domains to fine-tune BERT models that are able to capture cross-domain notions of relevance, and can be directly used for ranking news articles. Our simple neural ranking models achieve state-of-the-art effectiveness on three standard test collections.
The Challenges of Optimizing Machine Translation for Low Resource Cross-Language Information Retrieval
Authors: Constantine Lignos, Daniel Cohen, Yen-Chieh Lien, Pratik Mehta, W. Bruce Croft, Scott Miller
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Information Retrieval, Machine Translation, Retrieval, Translation

When performing cross-language information retrieval (CLIR) for lower-resourced languages, a common approach is to retrieve over the output of machine translation (MT). However, there is no established guidance on how to optimize the resulting MT-IR system. In this paper, we examine the relationship between the performance of MT systems and both neural and term frequency-based IR models to identify how CLIR performance can be best predicted from MT quality. We explore performance at varying amounts of MT training data, byte pair encoding (BPE) merge operations, and across two IR collections and retrieval models. We find that the choice of IR collection can substantially affect the predictive power of MT tuning decisions and evaluation, potentially introducing dissociations between MT-only and overall CLIR performance.
Leveraging Adjective-Noun Phrasing Knowledge for Comparison Relation Prediction in Text-to-SQL
Authors: Haoyan Liu, Lei Fang, Qian Liu, Bei Chen, Jian-Guang Lou, Zhoujun Li
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Relation, Relation Prediction, Text to SQL, Text-To-SQL

One key component in text-to-SQL is to predict the comparison relations between columns and their values. To the best of our knowledge, no existing models explicitly introduce external common knowledge to address this problem, thus their capabilities of predicting comparison relations are limited beyond training data. In this paper, we propose to leverage adjective-noun phrasing knowledge mined from the web to predict the comparison relations in text-to-SQL. Experimental results on both the original and the re-split Spider dataset show that our approach achieves significant improvement over state-of-the-art methods on comparison relation prediction.
Bridging the Defined and the Defining: Exploiting Implicit Lexical Semantic Relations in Definition Modeling
Authors: Koki Washio, Satoshi Sekine, Tsuneaki Kato
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Word Embeddings

Definition modeling includes acquiring word embeddings from dictionary definitions and generating definitions of words. While the meanings of defining words are important in dictionary definitions, it is crucial to capture the lexical semantic relations between defined words and defining words. However, thus far, the utilization of such relations has not been explored for definition modeling. In this paper, we propose definition modeling methods that use lexical semantic relations. To utilize implicit semantic relations in definitions, we use unsupervisedly obtained pattern-based word-pair embeddings that represent semantic relations of word pairs. Experimental results indicate that our methods improve the performance in learning embeddings from definitions, as well as definition generation.
SyntagNet: Challenging Supervised Word Sense Disambiguation with Lexical-Semantic Combinations
Authors: Marco Maru, Federico Scozzafava, Federico Martelli, Roberto Navigli
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Word Sense Disambiguation

Current research in knowledge-based Word Sense Disambiguation (WSD) indicates that performances depend heavily on the Lexical Knowledge Base (LKB) employed. This paper introduces SyntagNet, a novel resource consisting of manually disambiguated lexical-semantic combinations. By capturing sense distinctions evoked by syntagmatic relations, SyntagNet enables knowledge-based WSD systems to establish a new state of the art which challenges the hitherto unrivaled performances attained by supervised approaches. To the best of our knowledge, SyntagNet is the first large-scale manually-curated resource of this kind made available to the community (at http://syntagnet.org).
Fine-tune BERT with Sparse Self-Attention Mechanism
Authors: Baiyun Cui, Yingming Li, Ming Chen, Zhongfei Zhang
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Natural Language Inference, Question Answering, Sentiment Analysis

In this paper, we develop a novel Sparse Self-Attention Fine-tuning model (referred as SSAF) which integrates sparsity into self-attention mechanism to enhance the fine-tuning performance of BERT. In particular, sparsity is introduced into the self-attention by replacing softmax function with a controllable sparse transformation when fine-tuning with BERT. It enables us to learn a structurally sparse attention distribution, which leads to a more interpretable representation for the whole input. The proposed model is evaluated on sentiment analysis, question answering, and natural language inference tasks. The extensive experimental results across multiple datasets demonstrate its effectiveness and superiority to the baseline methods.
A Multi-Pairwise Extension of Procrustes Analysis for Multilingual Word Translation
Authors: Hagai Taitelbaum, Gal Chechik, Jacob Goldberger
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Translation, Word Translation

In this paper we present a novel approach to simultaneously representing multiple languages in a common space. Procrustes Analysis (PA) is commonly used to find the optimal orthogonal word mapping in the bilingual case. The proposed Multi Pairwise Procrustes Analysis (MPPA) is a natural extension of the PA algorithm to multilingual word mapping. Unlike previous PA extensions that require a k-way dictionary, this approach requires only pairwise bilingual dictionaries that are much easier to construct.
Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer
Authors: Yunli Wang, Yu Wu, Lili Mou, Zhoujun Li, WenHan Chao
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Formality Style Transfer, Style Transfer, Text Style Transfer

Formality text style transfer plays an important role in various NLP applications, such as non-native speaker assistants and child education. Early studies normalize informal sentences with rules, before statistical and neural models become a prevailing method in the field. While a rule-based system is still a common preprocessing step for formality style transfer in the neural era, it could introduce noise if we use the rules in a naive way such as data preprocessing. To mitigate this problem, we study how to harness rules into a state-of-the-art neural network that is typically pretrained on massive corpora. We propose three fine-tuning methods in this paper and achieve a new state-of-the-art on benchmark datasets
Multiple Text Style Transfer by using Word-level Conditional Generative Adversarial Network with Two-Phase Training
Authors: Chih-Te Lai, Yi-Te Hong, Hong-You Chen, Chi-Jen Lu, Shou-De Lin
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Attribute, Generative Adversarial Network, Style Transfer, Text Generation, Text Style Transfer

The objective of non-parallel text style transfer, or controllable text generation, is to alter specific attributes (e.g. sentiment, mood, tense, politeness, etc) of a given text while preserving its remaining attributes and content. Generative adversarial network (GAN) is a popular model to ensure the transferred sentences are realistic and have the desired target styles. However, training GAN often suffers from mode collapse problem, which causes that the transferred text is little related to the original text. In this paper, we propose a new GAN model with a word-level conditional architecture and a two-phase training procedure. By using a style-related condition architecture before generating a word, our model is able to maintain style-unrelated words while changing the others. By separating the training procedure into reconstruction and transfer phases, our model is able to learn a proper text generation process, which further improves the content preservation. We test our model on polarity sentiment transfer and multiple-attribute transfer tasks. The empirical results show that our model achieves comparable evaluation scores in both transfer accuracy and fluency but significantly outperforms other state-of-the-art models in content compatibility on three real-world datasets.
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition
Authors: Yufan Jiang, Chi Hu, Tong Xiao, Chunliang Zhang, Jingbo Zhu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Neural Architecture Search

In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing. In particular, we improve differentiable architecture search by removing the softmax-local constraint. Also, we apply differentiable NAS to named entity recognition (NER). It is the first time that differentiable NAS methods are adopted in NLP tasks other than language modeling. On both the PTB language modeling and CoNLL-2003 English NER data, our method outperforms strong baselines. It achieves a new state-of-the-art on the NER task.
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
Authors: Alham Fikri Aji, Kenneth Heafield, Nikolay Bogoychev
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, Translation

One way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients. However, doing so damages the gradient and degrades the model{'}s performance. Transformer models degrade dramatically while the impact on RNNs is smaller. We restore gradient quality by combining the compressed global gradient with the node{'}s locally computed uncompressed gradient. Neural machine translation experiments show that Transformer convergence is restored while RNNs converge faster. With our method, training on 4 nodes converges up to 1.5x as fast as with uncompressed gradients and scales 3.5x relative to single-node training.
Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis
Authors: Hiroki Ouchi, Jun Suzuki, Kentaro Inui
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Transductive Learning

In transductive learning, an unlabeled test set is used for model training. Although this setting deviates from the common assumption of a completely unseen test set, it is applicable in many real-world scenarios, wherein the texts to be processed are known in advance. However, despite its practical advantages, transductive learning is underexplored in natural language processing. Here we conduct an empirical study of transductive learning for neural models and demonstrate its utility in syntactic and semantic tasks. Specifically, we fine-tune language models (LMs) on an unlabeled test set to obtain test-set-specific word representations. Through extensive experiments, we demonstrate that despite its simplicity, transductive LM fine-tuning consistently improves state-of-the-art neural models in in-domain and out-of-domain settings.
Summary Cloze: A New Task for Content Selection in Topic-Focused Summarization
Authors: Daniel Deutsch, Dan Roth
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Sentence

A key challenge in topic-focused summarization is determining what information should be included in the summary, a problem known as content selection. In this work, we propose a new method for studying content selection in topic-focused summarization called the summary cloze task. The goal of the summary cloze task is to generate the next sentence of a summary conditioned on the beginning of the summary, a topic, and a reference document(s). The main challenge is deciding what information in the references is relevant to the topic and partial summary and should be included in the summary. Although the cloze task does not address all aspects of the traditional summarization problem, the more narrow scope of the task allows us to collect a large-scale datset of nearly 500k summary cloze instances from Wikipedia. We report experimental results on this new dataset using various extractive models and a two-step abstractive model that first extractively selects a small number of sentences and then abstractively summarizes them. Our results show that the topic and partial summary help the models identify relevant content, but the task remains a significant challenge.
Improving Latent Alignment in Text Summarization by Generalizing the Pointer Generator
Authors: Xiaoyu Shen, Yang Zhao, Hui Su, Dietrich Klakow
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Text Summarization, Word Alignment

Pointer Generators have been the de facto standard for modern summarization systems. However, this architecture faces two major drawbacks: Firstly, the pointer is limited to copying the exact words while ignoring possible inflections or abstractions, which restricts its power of capturing richer latent alignment. Secondly, the copy mechanism results in a strong bias towards extractive generations, where most sentences are produced by simply copying from the source text. In this paper, we address these problems by allowing the model to {``}edit{''} pointed tokens instead of always hard copying them. The editing is performed by transforming the pointed word vector into a target space with a learned relation embedding. On three large-scale summarization dataset, we show the model is able to (1) capture more latent alignment relations than exact word matches, (2) improve word alignment accuracy, allowing for better model interpretation and controlling, (3) generate higher-quality summaries validated by both qualitative and quantitative evaluations and (4) bring more abstraction to the generated summaries.
Improving Distantly-Supervised Relation Extraction with Joint Label Embedding
Authors: Linmei Hu, Luhao Zhang, Chuan Shi, Liqiang Nie, Weili Guan, Cheng Yang
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Knowledge Graphs, Relation, Relation Extraction, valid

Distantly-supervised relation extraction has proven to be effective to find relational facts from texts. However, the existing approaches treat labels as independent and meaningless one-hot vectors, which cause a loss of potential label information for selecting valid instances. In this paper, we propose a novel multi-layer attention-based model to improve relation extraction with joint label embedding. The model makes full use of both structural information from Knowledge Graphs and textual information from entity descriptions to learn label embeddings through gating integration while avoiding the imposed noise with an attention mechanism. Then the learned label embeddings are used as another atten- tion over the instances (whose embeddings are also enhanced with the entity descriptions) for improving relation extraction. Extensive experiments demonstrate that our model significantly outperforms state-of-the-art methods.