pwc-archive/papers-with-abstracts huggingface dataset:
Understanding and Improving Morphological Learning in the Neural Machine Translation Decoder
Authors: Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Stephan Vogel
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Decoder, Machine Translation, Multi-Task Learning, NMT, Translation

End-to-end training makes the neural machine translation (NMT) architecture simpler, yet elegant compared to traditional statistical machine translation (SMT). However, little is known about linguistic patterns of morphology, syntax and semantics learned during the training of NMT systems, and more importantly, which parts of the architecture are responsible for learning each of these phenomenon. In this paper we i) analyze how much morphology an NMT decoder learns, and ii) investigate whether injecting target morphology in the decoder helps it to produce better translations. To this end we present three methods: i) simultaneous translation, ii) joint-data learning, and iii) multi-task learning. Our results show that explicit morphological information helps the decoder learn target language morphology and improves the translation quality by 0.2{--}0.6 BLEU points.
Addressing Domain Adaptation for Chinese Word Segmentation with Global Recurrent Structure
Authors: Shen Huang, Xu sun, Houfeng Wang
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Articles, Chinese Word Segmentation, Domain Adaptation, Feature Engineering, Segmentation

Boundary features are widely used in traditional Chinese Word Segmentation (CWS) methods as they can utilize unlabeled data to help improve the Out-of-Vocabulary (OOV) word recognition performance. Although various neural network methods for CWS have achieved performance competitive with state-of-the-art systems, these methods, constrained by the domain and size of the training corpus, do not work well in domain adaptation. In this paper, we propose a novel BLSTM-based neural network model which incorporates a global recurrent structure designed for modeling boundary features dynamically. Experiments show that the proposed structure can effectively boost the performance of Chinese Word Segmentation, especially OOV-Recall, which brings benefits to domain adaptation. We achieved state-of-the-art results on 6 domains of CNKI articles, and competitive results to the best reported on the 4 domains of SIGHAN Bakeoff 2010 data.
Information Bottleneck Inspired Method For Chat Text Segmentation
Authors: S Vishal, Mohit Yadav, Lovekesh Vig, Gautam Shroff
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Representation Learning, Text Generation, Text Segmentation, Topic Models

We present a novel technique for segmenting chat conversations using the information bottleneck method (Tishby et al., 2000), augmented with sequential continuity constraints. Furthermore, we utilize critical non-textual clues such as time between two consecutive posts and people mentions within the posts. To ascertain the effectiveness of the proposed method, we have collected data from public Slack conversations and Fresco, a proprietary platform deployed inside our organization. Experiments demonstrate that the proposed method yields an absolute (relative) improvement of as high as 3.23{\%} (11.25{\%}). To facilitate future research, we are releasing manual annotations for segmentation on public Slack conversations.
A Computational Study on Word Meanings and Their Distributed Representations via Polymodal Embedding
Authors: Joohee Park, Sung-Hyon Myaeng
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Word Similarity

A distributed representation has become a popular approach to capturing a word meaning. Besides its success and practical value, however, questions arise about the relationships between a true word meaning and its distributed representation. In this paper, we examine such a relationship via polymodal embedding approach inspired by the theory that humans tend to use diverse sources in developing a word meaning. The result suggests that the existing embeddings lack in capturing certain aspects of word meanings which can be significantly improved by the polymodal approach. Also, we show distinct characteristics of different types of words (e.g. concreteness) via computational studies. Finally, we show our proposed embedding method outperforms the baselines in the word similarity measure tasks and the hypernym prediction tasks.
Geographical Evaluation of Word Embeddings
Authors: Michal Konkol, Tom{\'a}{\v{s}} Brychc{\'\i}n, Michal Nykl, Tom{\'a}{\v{s}} Hercig
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Machine Translation, Named Entity Recognition (NER), Position, Sentiment Analysis, Word Embeddings

Word embeddings are commonly compared either with human-annotated word similarities or through improvements in natural language processing tasks. We propose a novel principle which compares the information from word embeddings with reality. We implement this principle by comparing the information in the word embeddings with geographical positions of cities. Our evaluation linearly transforms the semantic space to optimally fit the real positions of cities and measures the deviation between the position given by word embeddings and the real position. A set of well-known word embeddings with state-of-the-art results were evaluated. We also introduce a visualization that helps with error analysis.
On Modeling Sense Relatedness in Multi-prototype Word Embedding
Authors: Yixin Cao, Jiaxin Shi, Juanzi Li, Zhiyuan Liu, Chengjiang Li
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Clustering, Language Modeling, Language Modelling, Named Entity Recognition (NER), Representation Learning, Word Sense Induction

To enhance the expression ability of distributional word representation learning model, many researchers tend to induce word senses through clustering, and learn multiple embedding vectors for each word, namely multi-prototype word embedding model. However, most related work ignores the relatedness among word senses which actually plays an important role. In this paper, we propose a novel approach to capture word sense relatedness in multi-prototype word embedding model. Particularly, we differentiate the original sense and extended senses of a word by introducing their global occurrence information and model their relatedness through the local textual context information. Based on the idea of fuzzy clustering, we introduce a random process to integrate these two types of senses and design two non-parametric methods for word sense induction. To make our model more scalable and efficient, we use an online joint learning framework extended from the Skip-gram model. The experimental results demonstrate that our model outperforms both conventional single-prototype embedding models and other multi-prototype embedding models, and achieves more stable performance when trained on smaller data.
Unsupervised Segmentation of Phoneme Sequences based on Pitman-Yor Semi-Markov Model using Phoneme Length Context
Authors: Ryu Takeda, Kazunori Komatani
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Segmentation, Spoken Dialogue Systems

Unsupervised segmentation of phoneme sequences is an essential process to obtain unknown words during spoken dialogues. In this segmentation, an input phoneme sequence without delimiters is converted into segmented sub-sequences corresponding to words. The Pitman-Yor semi-Markov model (PYSMM) is promising for this problem, but its performance degrades when it is applied to phoneme-level word segmentation. This is because of insufficient cues for the segmentation, e.g., homophones are improperly treated as single entries and their different contexts are also confused. We propose a phoneme-length context model for PYSMM to give a helpful cue at the phoneme-level and to predict succeeding segments more accurately. Our experiments showed that the peak performance with our context model outperformed those without such a context model by 0.045 at most in terms of F-measures of estimated segmentation.
Coordination Boundary Identification with Similarity and Replaceability
Authors: Hiroki Teranishi, Hiroyuki Shindo, Yuji Matsumoto
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Boundary Detection

We propose a neural network model for coordination boundary detection. Our method relies on the two common properties - similarity and replaceability in conjuncts - in order to detect both similar pairs of conjuncts and dissimilar pairs of conjuncts. The model improves identification of clause-level coordination using bidirectional RNNs incorporating two properties as features. We show that our model outperforms the existing state-of-the-art methods on the coordination annotated Penn Treebank and Genia corpus without any syntactic information from parsers.
Turning Distributional Thesauri into Word Vectors for Synonym Extraction and Expansion
Authors: Olivier Ferret
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Graph Embedding, Semantic Textual Similarity, Word Embeddings

In this article, we propose to investigate a new problem consisting in turning a distributional thesaurus into dense word vectors. We propose more precisely a method for performing such task by associating graph embedding and distributed representation adaptation. We have applied and evaluated it for English nouns at a large scale about its ability to retrieve synonyms. In this context, we have also illustrated the interest of the developed method for three different tasks: the improvement of already existing word embeddings, the fusion of heterogeneous representations and the expansion of synsets.
Training Word Sense Embeddings With Lexicon-based Regularization
Authors: Luis Nieto-Pi{\~n}a, Richard Johansson
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Word Embeddings, Word Sense Disambiguation

We propose to improve word sense embeddings by enriching an automatic corpus-based method with lexicographic data. Information from a lexicon is introduced into the learning algorithm{'}s objective function through a regularizer. The incorporation of lexicographic data yields embeddings that are able to reflect expert-defined word senses, while retaining the robustness, high quality, and coverage of automatic corpus-based methods. These properties are observed in a manual inspection of the semantic clusters that different degrees of regularizer strength create in the vector space. Moreover, we evaluate the sense embeddings in two downstream applications: word sense disambiguation and semantic frame prediction, where they outperform simpler approaches. Our results show that a corpus-based model balanced with lexicographic data learns better representations and improve their performance in downstream tasks.
Learning How to Simplify From Explicit Labeling of Complex-Simplified Text Pairs
Authors: Fern Alva-Manchego, o, Joachim Bingel, Gustavo Paetzold, Carolina Scarton, Lucia Specia
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Machine Translation, Sentence, Sentence Compression, Text Simplification

Current research in text simplification has been hampered by two central problems: (i) the small amount of high-quality parallel simplification data available, and (ii) the lack of explicit annotations of simplification operations, such as deletions or substitutions, on existing data. While the recently introduced Newsela corpus has alleviated the first problem, simplifications still need to be learned directly from parallel text using black-box, end-to-end approaches rather than from explicit annotations. These complex-simple parallel sentence pairs often differ to such a high degree that generalization becomes difficult. End-to-end models also make it hard to interpret what is actually learned from data. We propose a method that decomposes the task of TS into its sub-problems. We devise a way to automatically identify operations in a parallel corpus and introduce a sequence-labeling approach based on these annotations. Finally, we provide insights on the types of transformations that different approaches can model.
Domain-Adaptable Hybrid Generation of RDF Entity Descriptions
Authors: Or Biran, Kathleen McKeown
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Diversity, Domain Adaptation

RDF ontologies provide structured data on entities in many domains and continue to grow in size and diversity. While they can be useful as a starting point for generating descriptions of entities, they often miss important information about an entity that cannot be captured as simple relations. In addition, generic approaches to generation from RDF cannot capture the unique style and content of specific domains. We describe a framework for hybrid generation of entity descriptions, which combines generation from RDF data with text extracted from a corpus, and extracts unique aspects of the domain from the corpus to create domain-specific generation systems. We show that each component of our approach significantly increases the satisfaction of readers with the text across multiple applications and domains.
ES-LDA: Entity Summarization using Knowledge-based Topic Modeling
Authors: Seyedamin Pouriyeh, Mehdi Allahyari, Krzysztof Kochut, Gong Cheng, Hamid Reza Arabnia
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Document Summarization

With the advent of the Internet, the amount of Semantic Web documents that describe real-world entities and their inter-links as a set of statements have grown considerably. These descriptions are usually lengthy, which makes the utilization of the underlying entities a difficult task. Entity summarization, which aims to create summaries for real-world entities, has gained increasing attention in recent years. In this paper, we propose a probabilistic topic model, ES-LDA, that combines prior knowledge with statistical learning techniques within a single framework to create more reliable and representative summaries for entities. We demonstrate the effectiveness of our approach by conducting extensive experiments and show that our model outperforms the state-of-the-art techniques and enhances the quality of the entity summaries.
Procedural Text Generation from an Execution Video
Authors: Atsushi Ushiku, Hayato Hashimoto, Atsushi Hashimoto, Shinsuke Mori
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Object Recognition, Text Generation, Video Captioning

In recent years, there has been a surge of interest in automatically describing images or videos in a natural language. These descriptions are useful for image/video search, etc. In this paper, we focus on procedure execution videos, in which a human makes or repairs something and propose a method for generating procedural texts from them. Since video/text pairs available are limited in size, the direct application of end-to-end deep learning is not feasible. Thus we propose to train Faster R-CNN network for object recognition and LSTM for text generation and combine them at run time. We took pairs of recipe and cooking video, generated a recipe from a video, and compared it with the original recipe. The experimental results showed that our method can produce a recipe as accurate as the state-of-the-art scene descriptions.
Text Sentiment Analysis based on Fusion of Structural Information and Serialization Information
Authors: Ling Gan, Houyu Gong
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Opinion Mining, Sentiment Analysis

Tree-structured Long Short-Term Memory (Tree-LSTM) has been proved to be an effective method in the sentiment analysis task. It extracts structural information on text, and uses Long Short-Term Memory (LSTM) cell to prevent gradient vanish. However, though combining the LSTM cell, it is still a kind of model that extracts the structural information and almost not extracts serialization information. In this paper, we propose three new models in order to combine those two kinds of information: the structural information generated by the Constituency Tree-LSTM and the serialization information generated by Long-Short Term Memory neural network. Our experiments show that combining those two kinds of information can give contributes to the performance of the sentiment analysis task compared with the single Constituency Tree-LSTM model and the LSTM model.
Length, Interchangeability, and External Knowledge: Observations from Predicting Argument Convincingness
Authors: Peter Potash, Robin Bhattacharya, Anna Rumshisky
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Semantic Composition

In this work, we provide insight into three key aspects related to predicting argument convincingness. First, we explicitly display the power that text length possesses for predicting convincingness in an unsupervised setting. Second, we show that a bag-of-words embedding model posts state-of-the-art on a dataset of arguments annotated for convincingness, outperforming an SVM with numerous hand-crafted features as well as recurrent neural network models that attempt to capture semantic composition. Finally, we assess the feasibility of integrating external knowledge when predicting convincingness, as arguments are often more convincing when they contain abundant information and facts. We finish by analyzing the correlations between the various models we propose.
Exploiting Document Level Information to Improve Event Detection via Recurrent Neural Networks
Authors: Shaoyang Duan, Ruifang He, Wenli Zhao
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Event Detection, Sentence

This paper tackles the task of event detection, which involves identifying and categorizing events. The previous work mainly exist two problems: (1) the traditional feature-based methods apply cross-sentence information, yet need taking a large amount of human effort to design complicated feature sets and inference rules; (2) the representation-based methods though overcome the problem of manually extracting features, while just depend on local sentence representation. Considering local sentence context is insufficient to resolve ambiguities in identifying particular event types, therefore, we propose a novel document level Recurrent Neural Networks (DLRNN) model, which can automatically extract cross-sentence clues to improve sentence level event detection without designing complex reasoning rules. Experiment results show that our approach outperforms other state-of-the-art methods on ACE 2005 dataset without external knowledge base.
Embracing Non-Traditional Linguistic Resources for Low-resource Language Name Tagging
Authors: Boliang Zhang, Di Lu, Xiaoman Pan, Ying Lin, Halidanmu Abudukelimu, Heng Ji, Kevin Knight
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Relation Classification, Word Embeddings

Current supervised name tagging approaches are inadequate for most low-resource languages due to the lack of annotated data and actionable linguistic knowledge. All supervised learning methods (including deep neural networks (DNN)) are sensitive to noise and thus they are not quite portable without massive clean annotations. We found that the F-scores of DNN-based name taggers drop rapidly (20{\%}-30{\%}) when we replace clean manual annotations with noisy annotations in the training data. We propose a new solution to incorporate many non-traditional language universal resources that are readily available but rarely explored in the Natural Language Processing (NLP) community, such as the World Atlas of Linguistic Structure, CIA names, PanLex and survival guides. We acquire and encode various types of non-traditional linguistic resources into a DNN name tagger. Experiments on three low-resource languages show that feeding linguistic knowledge can make DNN significantly more robust to noise, achieving 8{\%}-22{\%} absolute F-score gains on name tagging without using any human annotation
NMT or SMT: Case Study of a Narrow-domain English-Latvian Post-editing Project
Authors: Inguna Skadi{\c{n}}a, M{\=a}rcis Pinnis
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Machine Translation, NMT, Translation

The recent technological shift in machine translation from statistical machine translation (SMT) to neural machine translation (NMT) raises the question of the strengths and weaknesses of NMT. In this paper, we present an analysis of NMT and SMT systems{'} outputs from narrow domain English-Latvian MT systems that were trained on a rather small amount of data. We analyze post-edits produced by professional translators and manually annotated errors in these outputs. Analysis of post-edits allowed us to conclude that both approaches are comparably successful, allowing for an increase in translators{'} productivity, with the NMT system showing slightly worse results. Through the analysis of annotated errors, we found that NMT translations are more fluent than SMT translations. However, errors related to accuracy, especially, mistranslation and omission errors, occur more often in NMT outputs. The word form errors, that characterize the morphological richness of Latvian, are frequent for both systems, but slightly fewer in NMT outputs.
Identifying Usage Expression Sentences in Consumer Product Reviews
Authors: Shibamouli Lahiri, V.G.Vinod Vydiswaran, Rada Mihalcea
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Classification, General Classification

In this paper we introduce the problem of identifying usage expression sentences in a consumer product review. We create a human-annotated gold standard dataset of 565 reviews spanning five distinct product categories. Our dataset consists of more than 3,000 annotated sentences. We further introduce a classification system to label sentences according to whether or not they describe some {``}usage{''}. The system combines lexical, syntactic, and semantic features in a product-agnostic fashion to yield good classification performance. We show the effectiveness of our approach using importance ranking of features, error analysis, and cross-product classification experiments.
Between Reading Time and Syntactic/Semantic Categories
Authors: Masayuki Asahara, Sachi Kato
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11

This article presents a contrastive analysis between reading time and syntactic/semantic categories in Japanese. We overlaid the reading time annotation of BCCWJ-EyeTrack and a syntactic/semantic category information annotation on the {`}Balanced Corpus of Contemporary Written Japanese{'}. Statistical analysis based on a mixed linear model showed that verbal phrases tend to have shorter reading times than adjectives, adverbial phrases, or nominal phrases. The results suggest that the preceding phrases associated with the presenting phrases promote the reading process to shorten the gazing time.
WiNER: A Wikipedia Annotated Corpus for Named Entity Recognition
Authors: Abbas Ghaddar, Phillippe Langlais
Date: 2017-11-01 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

We revisit the idea of mining Wikipedia in order to generate named-entity annotations. We propose a new methodology that we applied to English Wikipedia to build WiNER, a large, high quality, annotated corpus. We evaluate its usefulness on 6 NER tasks, comparing 4 popular state-of-the art approaches. We show that LSTM-CRF is the approach that benefits the most from our corpus. We report impressive gains with this model when using a small portion of WiNER on top of the CONLL training material. Last, we propose a simple but efficient method for exploiting the full range of WiNER, leading to further improvements.