pwc-archive/papers-with-abstracts huggingface dataset:
A Multilingual Topic Model for Learning Weighted Topic Links Across Corpora with Low Comparability
Authors: Weiwei Yang, Jordan Boyd-Graber, Philip Resnik
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Topic Models

Multilingual topic models (MTMs) learn topics on documents in multiple languages. Past models align topics across languages by implicitly assuming the documents in different languages are highly comparable, often a false assumption. We introduce a new model that does not rely on this assumption, particularly useful in important low-resource language scenarios. Our MTM learns weighted topic links and connects cross-lingual topics only when the dominant words defining them are similar, outperforming LDA and previous MTMs in classification tasks using documents{'} topic posteriors as features. It also learns coherent topics on documents with low comparability.
Measure Country-Level Socio-Economic Indicators with Streaming News: An Empirical Study
Authors: Bonan Min, Xiaoxi Zhao
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Socio-economic conditions are difficult to measure. For example, the U.S. Bureau of Labor Statistics needs to conduct large-scale household surveys regularly to track the unemployment rate, an indicator widely used by economists and policymakers. We argue that events reported in streaming news can be used as {``}micro-sensors{''} for measuring socio-economic conditions. Similar to collecting surveys and then counting answers, it is possible to measure a socio-economic indicator by counting related events. In this paper, we propose Event-Centric Indicator Measure (ECIM), a novel approach to measure socio-economic indicators with events. We empirically demonstrate strong correlation between ECIM values to several representative indicators in socio-economic research.
Towards Extracting Medical Family History from Natural Language Interactions: A New Dataset and Baselines
Authors: Mahmoud Azab, Stephane Dadian, Vivi Nastase, Larry An, Rada Mihalcea
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Relation Extraction

We introduce a new dataset consisting of natural language interactions annotated with medical family histories, obtained during interactions with a genetic counselor and through crowdsourcing, following a questionnaire created by experts in the domain. We describe the data collection process and the annotations performed by medical professionals, including illness and personal attributes (name, age, gender, family relationships) for the patient and their family members. An initial system that performs argument identification and relation extraction shows promising results {--} average F-score of 0.87 on complex sentences on the targeted relations.
Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue
Authors: Chenguang Zhu, Michael Zeng, Xuedong Huang
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Language Modeling, Language Modelling, Multi-Task Learning, Text Generation

In task-oriented dialogues, Natural Language Generation (NLG) is the final yet crucial step to produce user-facing system utterances. The result of NLG is directly related to the perceived quality and usability of a dialogue system. While most existing systems provide semantically correct responses given goals to present, they struggle to match the variation and fluency in the human language. In this paper, we propose a novel multi-task learning framework, NLG-LM, for natural language generation. In addition to generating high-quality responses conveying the required information, it also explicitly targets for naturalness in generated responses via an unconditioned language model. This can significantly improve the learning of style and variation in human language. Empirical results show that this multi-task learning framework outperforms previous models across multiple datasets. For example, it improves the previous best BLEU score on the E2E-NLG dataset by 2.2{\%}, and on the Laptop dataset by 6.1{\%}.
Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation
Authors: Min Zeng, Yisen Wang, Yuan Luo
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Decoder, Dialogue Generation, Sentence

Variational encoder-decoders have achieved well-recognized performance in the dialogue generation task. Existing works simply assume the Gaussian priors of the latent variable, which are incapable of representing complex latent variables effectively. To address the issues, we propose to use the Dirichlet distribution with flexible structures to characterize the latent variables in place of the traditional Gaussian distribution, called Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder model (Dir-VHRED). Based on which, we further find that there is redundancy among the dimensions of latent variable, and the lengths and sentence patterns of the responses can be strongly correlated to each dimension of the latent variable. Therefore, controllable responses can be generated through specifying the value of each dimension of the latent variable. Experimental results on benchmarks show that our proposed Dir-VHRED yields substantial improvements on negative log-likelihood, word-embedding-based and human evaluations.
Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling
Authors: Bo-Hsiang Tseng, Marek Rei, Pawe{\l} Budzianowski, Richard Turner, Bill Byrne, Anna Korhonen
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels. However, collecting these annotations is expensive and time-consuming, holding back development in the area of dialogue modelling. In this paper, we investigate semi-supervised learning methods that are able to reduce the amount of required intermediate labelling. We find that by leveraging un-annotated data instead, the amount of turn-level annotations of dialogue state can be significantly reduced when building a neural dialogue system. Our analysis on the MultiWOZ corpus, covering a range of domains and topics, finds that annotations can be reduced by up to 30{\%} while maintaining equivalent system performance. We also describe and evaluate the first end-to-end dialogue model created for the MultiWOZ corpus.
A Progressive Model to Enable Continual Learning for Semantic Slot Filling
Authors: Yilin Shen, Xiangyu Zeng, Hongxia Jin
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Continual Learning, slot-filling, Slot Filling, Spoken Language Understanding

Semantic slot filling is one of the major tasks in spoken language understanding (SLU). After a slot filling model is trained on precollected data, it is crucial to continually improve the model after deployment to learn users{'} new expressions. As the data amount grows, it becomes infeasible to either store such huge data and repeatedly retrain the model on all data or fine tune the model only on new data without forgetting old expressions. In this paper, we introduce a novel progressive slot filling model, ProgModel. ProgModel consists of a novel context gate that transfers previously learned knowledge to a small size expanded component; and meanwhile enables this new component to be fast trained to learn from new data. As such, ProgModel learns the new knowledge by only using new data at each time and meanwhile preserves the previously learned expressions. Our experiments show that ProgModel needs much less training time and smaller model size to outperform various model fine tuning competitors by up to 4.24{\%} and 3.03{\%} on two benchmark datasets.
Sampling Matters! An Empirical Study of Negative Sampling Strategies for Learning of Matching Models in Retrieval-based Dialogue Systems
Authors: Jia Li, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, Rui Yan
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Conversational Response Selection, Retrieval

We study how to sample negative examples to automatically construct a training set for effective model learning in retrieval-based dialogue systems. Following an idea of dynamically adapting negative examples to matching models in learning, we consider four strategies including minimum sampling, maximum sampling, semi-hard sampling, and decay-hard sampling. Empirical studies on two benchmarks with three matching models indicate that compared with the widely used random sampling strategy, although the first two strategies lead to performance drop, the latter two ones can bring consistent improvement to the performance of all the models on both benchmarks.
Multilingual word translation using auxiliary languages
Authors: Hagai Taitelbaum, Gal Chechik, Jacob Goldberger
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Translation, Word Translation

Current multilingual word translation methods are focused on jointly learning mappings from each language to a shared space. The actual translation, however, is still performed as an isolated bilingual task. In this study we propose a multilingual translation procedure that uses all the learned mappings to translate a word from one language to another. For each source word, we first search for the most relevant auxiliary languages. We then use the translations to these languages to form an improved representation of the source word. Finally, this representation is used for the actual translation to the target language. Experiments on a standard multilingual word translation benchmark demonstrate that our model outperforms state of the art results.
Vecalign: Improved Sentence Alignment in Linear Time and Space
Authors: Brian Thompson, Philipp Koehn
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, Sentence, Sentence Embeddings, Translation

We introduce Vecalign, a novel bilingual sentence alignment method which is linear in time and space with respect to the number of sentences being aligned and which requires only bilingual sentence embeddings. On a standard German{--}French test set, Vecalign outperforms the previous state-of-the-art method (which has quadratic time complexity and requires a machine translation system) by 5 F1 points. It substantially outperforms the popular Hunalign toolkit at recovering Bible verse alignments in medium- to low-resource language pairs, and it improves downstream MT quality by 1.7 and 1.6 BLEU in Sinhala-English and Nepali-English, respectively, compared to the Hunalign-based Paracrawl pipeline.
Recurrent Positional Embedding for Neural Machine Translation
Authors: Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, Translation

In the Transformer network architecture, positional embeddings are used to encode order dependencies into the input representation. However, this input representation only involves static order dependencies based on discrete numerical information, that is, are independent of word content. To address this issue, this work proposes a recurrent positional embedding approach based on word vector. In this approach, these recurrent positional embeddings are learned by a recurrent neural network, encoding word content-based order dependencies into the input representation. They are then integrated into the existing multi-head self-attention model as independent heads or part of each head. The experimental results revealed that the proposed approach improved translation performance over that of the state-of-the-art Transformer baseline in WMT{'}14 English-to-German and NIST Chinese-to-English translation tasks.
Investigating the Effectiveness of BPE: The Power of Shorter Sequences
Authors: Matthias Gall{\'e}
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, Translation

Byte-Pair Encoding (BPE) is an unsupervised sub-word tokenization technique, commonly used in neural machine translation and other NLP tasks. Its effectiveness makes it a de facto standard, but the reasons for this are not well understood. We link BPE to the broader family of dictionary-based compression algorithms and compare it with other members of this family. Our experiments across datasets, language pairs, translation models, and vocabulary size show that - given a fixed vocabulary size budget - the fewer tokens an algorithm needs to cover the test set, the better the translation (as measured by BLEU).
HABLex: Human Annotated Bilingual Lexicons for Experiments in Machine Translation
Authors: Brian Thompson, Rebecca Knowles, Xuan Zhang, Huda Khayrallah, Kevin Duh, Philipp Koehn
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Machine Translation, Translation

Bilingual lexicons are valuable resources used by professional human translators. While these resources can be easily incorporated in statistical machine translation, it is unclear how to best do so in the neural framework. In this work, we present the HABLex dataset, designed to test methods for bilingual lexicon integration into neural machine translation. Our data consists of human generated alignments of words and phrases in machine translation test sets in three language pairs (Russian-English, Chinese-English, and Korean-English), resulting in clean bilingual lexicons which are well matched to the reference. We also present two simple baselines - constrained decoding and continued training - and an improvement to continued training to address overfitting.
Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation
Authors: Raj Dabre, Atsushi Fujita, Chenhui Chu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Low Resource Neural Machine Translation, Low-Resource Neural Machine Translation, Machine Translation, NMT, Transfer Learning, Translation

This paper highlights the impressive utility of multi-parallel corpora for transfer learning in a one-to-many low-resource neural machine translation (NMT) setting. We report on a systematic comparison of multistage fine-tuning configurations, consisting of (1) pre-training on an external large (209k{--}440k) parallel corpus for English and a helping target language, (2) mixed pre-training or fine-tuning on a mixture of the external and low-resource (18k) target parallel corpora, and (3) pure fine-tuning on the target parallel corpora. Our experiments confirm that multi-parallel corpora are extremely useful despite their scarcity and content-wise redundancy thus exhibiting the true power of multilingualism. Even when the helping target language is not one of the target languages of our concern, our multistage fine-tuning can give 3{--}9 BLEU score gains over a simple one-to-one model.
A Regularization-based Framework for Bilingual Grammar Induction
Authors: Yong Jiang, Wenjuan Han, Kewei Tu
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11

Grammar induction aims to discover syntactic structures from unannotated sentences. In this paper, we propose a framework in which the learning process of the grammar model of one language is influenced by knowledge from the model of another language. Unlike previous work on multilingual grammar induction, our approach does not rely on any external resource, such as parallel corpora, word alignments or linguistic phylogenetic trees. We propose three regularization methods that encourage similarity between model parameters, dependency edge scores, and parse trees respectively. We deploy our methods on a state-of-the-art unsupervised discriminative parser and evaluate it on both transfer grammar induction and bilingual grammar induction. Empirical results on multiple languages show that our methods outperform strong baselines.
Korean Morphological Analysis with Tied Sequence-to-Sequence Multi-Task Model
Authors: Hyun-Je Song, Seong-Bae Park
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Morphological Analysis, POS, POS Tagging

Korean morphological analysis has been considered as a sequence of morpheme processing and POS tagging. Thus, a pipeline model of the tasks has been adopted widely by previous studies. However, the model has a problem that it cannot utilize interactions among the tasks. This paper formulates Korean morphological analysis as a combination of the tasks and presents a tied sequence-to-sequence multi-task model for training the two tasks simultaneously without any explicit regularization. The experiments prove the proposed model achieves the state-of-the-art performance.
Cross-lingual Transfer Learning with Data Selection for Large-Scale Spoken Language Understanding
Authors: Quynh Do, Judith Gaspers
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Cross-Lingual Transfer, Language Modeling, Language Modelling, Spoken Language Understanding, Transfer Learning

A typical cross-lingual transfer learning approach boosting model performance on a language is to pre-train the model on all available supervised data from another language. However, in large-scale systems this leads to high training times and computational requirements. In addition, characteristic differences between the source and target languages raise a natural question of whether source data selection can improve the knowledge transfer. In this paper, we address this question and propose a simple but effective language model based source-language data selection method for cross-lingual transfer learning in large-scale spoken language understanding. The experimental results show that with data selection i) source data and hence training speed is reduced significantly and ii) model performance is improved.
WSLLN:Weakly Supervised Natural Language Localization Networks
Authors: Mingfei Gao, Larry Davis, Richard Socher, Caiming Xiong
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Sentence

We propose weakly supervised language localization networks (WSLLN) to detect events in long, untrimmed videos given language queries. To learn the correspondence between visual segments and texts, most previous methods require temporal coordinates (start and end times) of events for training, which leads to high costs of annotation. WSLLN relieves the annotation burden by training with only video-sentence pairs without accessing to temporal locations of events. With a simple end-to-end structure, WSLLN measures segment-text consistency and conducts segment selection (conditioned on the text) simultaneously. Results from both are merged and optimized as a video-sentence matching problem. Experiments on ActivityNet Captions and DiDeMo demonstrate that WSLLN achieves state-of-the-art performance.
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders
Authors: Andrew Drozdov, Patrick Verga, Yi-Pei Chen, Mohit Iyyer, Andrew McCallum
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Constituency Parsing

Understanding text often requires identifying meaningful constituent spans such as noun phrases and verb phrases. In this work, we show that we can effectively recover these types of labels using the learned phrase vectors from deep inside-outside recursive autoencoders (DIORA). Specifically, we cluster span representations to induce span labels. Additionally, we improve the model{'}s labeling accuracy by integrating latent code learning into the training procedure. We evaluate this approach empirically through unsupervised labeled constituency parsing. Our method outperforms ELMo and BERT on two versions of the Wall Street Journal (WSJ) dataset and is competitive to prior work that requires additional human annotations, improving over a previous state-of-the-art system that depends on ground-truth part-of-speech tags by 5 absolute F1 points (19{\%} relative error reduction).
Span-based Hierarchical Semantic Parsing for Task-Oriented Dialog
Authors: Panupong Pasupat, Sonal Gupta, M, Karishma yam, Rushin Shah, Mike Lewis, Luke Zettlemoyer
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Decoder, Semantic Parsing, valid

We propose a semantic parser for parsing compositional utterances into Task Oriented Parse (TOP), a tree representation that has intents and slots as labels of nesting tree nodes. Our parser is span-based: it scores labels of the tree nodes covering each token span independently, but then decodes a valid tree globally. In contrast to previous sequence decoding approaches and other span-based parsers, we (1) improve the training speed by removing the need to run the decoder at training time; and (2) introduce edge scores, which model relations between parent and child labels, to mitigate the independence assumption between node labels and improve accuracy. Our best parser outperforms previous methods on the TOP dataset of mixed-domain task-oriented utterances in both accuracy and training speed.
Hierarchical Modeling of Global Context for Document-Level Neural Machine Translation
Authors: Xin Tan, Longyin Zhang, Deyi Xiong, Guodong Zhou
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Document Level Machine Translation, Machine Translation, NMT, Sentence, Translation

Document-level machine translation (MT) remains challenging due to the difficulty in efficiently using document context for translation. In this paper, we propose a hierarchical model to learn the global context for document-level neural machine translation (NMT). This is done through a sentence encoder to capture intra-sentence dependencies and a document encoder to model document-level inter-sentence consistency and coherence. With this hierarchical architecture, we feedback the extracted global document context to each word in a top-down fashion to distinguish different translations of a word according to its specific surrounding context. In addition, since large-scale in-domain document-level parallel corpora are usually unavailable, we use a two-step training strategy to take advantage of a large-scale corpus with out-of-domain parallel sentence pairs and a small-scale corpus with in-domain parallel document pairs to achieve the domain adaptability. Experimental results on several benchmark corpora show that our proposed model can significantly improve document-level translation performance over several strong NMT baselines.
Asking Clarification Questions in Knowledge-Based Question Answering
Authors: Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, Xu sun
Date: 2019-11-01 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Question Answering, Question Generation, Question-Generation

The ability to ask clarification questions is essential for knowledge-based question answering (KBQA) systems, especially for handling ambiguous phenomena. Despite its importance, clarification has not been well explored in current KBQA systems. Further progress requires supervised resources for training and evaluation, and powerful models for clarification-related text understanding and generation. In this paper, we construct a new clarification dataset, CLAQUA, with nearly 40K open-domain examples. The dataset supports three serial tasks: given a question, identify whether clarification is needed; if yes, generate a clarification question; then predict answers base on external user feedback. We provide representative baselines for these tasks and further introduce a coarse-to-fine model for clarification question generation. Experiments show that the proposed model achieves better performance than strong baselines. The further analysis demonstrates that our dataset brings new challenges and there still remain several unsolved problems, like reasonable automatic evaluation metrics for clarification question generation and powerful models for handling entity sparsity.