pwc-archive/papers-with-abstracts huggingface dataset:
Sequence Classification with Human Attention
Authors: Maria Barrett, Joachim Bingel, Nora Hollenstein, Marek Rei, Anders S{\o}gaard
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Abusive Language, Classification, General Classification, Grammatical Error Detection, Inductive Bias, Sentiment Analysis

Learning attention functions requires large volumes of data, but many NLP tasks simulate human behavior, and in this paper, we show that human attention really does provide a good inductive bias on many attention functions in NLP. Specifically, we use estimated human attention derived from eye-tracking corpora to regularize attention functions in recurrent neural networks. We show substantial improvements across a range of tasks, including sentiment analysis, grammatical error detection, and detection of abusive language.
Learning to Actively Learn Neural Machine Translation
Authors: Ming Liu, Wray Buntine, Gholamreza Haffari
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Active Learning, Imitation Learning, Machine Translation, Sentence, Small Data Image Classification, Translation

Traditional active learning (AL) methods for machine translation (MT) rely on heuristics. However, these heuristics are limited when the characteristics of the MT problem change due to e.g. the language pair or the amount of the initial bitext. In this paper, we present a framework to learn sentence selection strategies for neural MT. We train the AL query strategy using a high-resource language-pair based on AL simulations, and then transfer it to the low-resource language-pair of interest. The learned query strategy capitalizes on the shared characteristics between the language pairs to make an effective use of the AL budget. Our experiments on three language-pairs confirms that our method is more effective than strong heuristic-based methods in various conditions, including cold-start and warm-start as well as small and extremely small data conditions.
Vectorial Semantic Spaces Do Not Encode Human Judgments of Intervention Similarity
Authors: Paola Merlo, Francesco Ackermann
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Word Embeddings

Despite their practical success and impressive performances, neural-network-based and distributed semantics techniques have often been criticized as they remain fundamentally opaque and difficult to interpret. In a vein similar to recent pieces of work investigating the linguistic abilities of these representations, we study another core, defining property of language: the property of long-distance dependencies. Human languages exhibit the ability to interpret discontinuous elements distant from each other in the string as if they were adjacent. This ability is blocked if a similar, but extraneous, element intervenes between the discontinuous components. We present results that show, under exhaustive and precise conditions, that one kind of word embeddings and the similarity spaces they define do not encode the properties of intervention similarity in long-distance dependencies, and that therefore they fail to represent this core linguistic notion.
Resources to Examine the Quality of Word Embedding Models Trained on n-Gram Data
Authors: {\'A}bel Elekes, Adrian Englhardt, Martin Sch{\"a}ler, Klemens B{\"o}hm
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Semantic Textual Similarity, Sentiment Analysis, Word Embeddings

Word embeddings are powerful tools that facilitate better analysis of natural language. However, their quality highly depends on the resource used for training. There are various approaches relying on n-gram corpora, such as the Google n-gram corpus. However, n-gram corpora only offer a small window into the full text {--} 5 words for the Google corpus at best. This gives way to the concern whether the extracted word semantics are of high quality. In this paper, we address this concern with two contributions. First, we provide a resource containing 120 word-embedding models {--} one of the largest collection of embedding models. Furthermore, the resource contains the n-gramed versions of all used corpora, as well as our scripts used for corpus generation, model generation and evaluation. Second, we define a set of meaningful experiments allowing to evaluate the aforementioned quality differences. We conduct these experiments using our resource to show its usage and significance. The evaluation results confirm that one generally can expect high quality for n-grams with n {\textgreater} 3.
Linguistically-Based Deep Unstructured Question Answering
Authors: Ahmad Aghaebrahimian
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Information Retrieval, Question Answering, Sentence

In this paper, we propose a new linguistically-based approach to answering non-factoid open-domain questions from unstructured data. First, we elaborate on an architecture for textual encoding based on which we introduce a deep end-to-end neural model. This architecture benefits from a bilateral attention mechanism which helps the model to focus on a question and the answer sentence at the same time for phrasal answer extraction. Second, we feed the output of a constituency parser into the model directly and integrate linguistic constituents into the network to help it concentrate on chunks of an answer rather than on its single words for generating more natural output. By optimizing this architecture, we managed to obtain near-to-human-performance results and competitive to a state-of-the-art system on SQuAD and MS-MARCO datasets respectively.
DIMSIM: An Accurate Chinese Phonetic Similarity Algorithm Based on Learned High Dimensional Encoding
Authors: Min Li, Marina Danilevsky, Sara Noeman, Yunyao Li
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Spelling Correction

Phonetic similarity algorithms identify words and phrases with similar pronunciation which are used in many natural language processing tasks. However, existing approaches are designed mainly for Indo-European languages and fail to capture the unique properties of Chinese pronunciation. In this paper, we propose a high dimensional encoded phonetic similarity algorithm for Chinese, DIMSIM. The encodings are learned from annotated data to separately map initial and final phonemes into n-dimensional coordinates. Pinyin phonetic similarities are then calculated by aggregating the similarities of initial, final and tone. DIMSIM demonstrates a 7.5X improvement on mean reciprocal rank over the state-of-the-art phonetic similarity approaches.
Challenge or Empower: Revisiting Argumentation Quality in a News Editorial Corpus
Authors: Roxanne El Baff, Henning Wachsmuth, Khalid Al-Khatib, Benno Stein
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Argument Mining

News editorials are said to shape public opinion, which makes them a powerful tool and an important source of political argumentation. However, rarely do editorials change anyone{'}s stance on an issue completely, nor do they tend to argue explicitly (but rather follow a subtle rhetorical strategy). So, what does argumentation quality mean for editorials then? We develop the notion that an effective editorial challenges readers with opposing stance, and at the same time empowers the arguing skills of readers that share the editorial{'}s stance {---} or even challenges both sides. To study argumentation quality based on this notion, we introduce a new corpus with 1000 editorials from the New York Times, annotated for their perceived effect along with the annotators{'} political orientations. Analyzing the corpus, we find that annotators with different orientation disagree on the effect significantly. While only 1{\%} of all editorials changed anyone{'}s stance, more than 5{\%} meet our notion. We conclude that our corpus serves as a suitable resource for studying the argumentation quality of news editorials.
Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP)
Authors: Trevor Cohen, Dominic Widdows
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Retrieval, Word Embeddings

Word order is clearly a vital part of human language, but it has been used comparatively lightly in distributional vector models. This paper presents a new method for incorporating word order information into word vector embedding models by combining the benefits of permutation-based order encoding with the more recent method of skip-gram with negative sampling. The new method introduced here is called Embeddings Augmented by Random Permutations (EARP). It operates by applying permutations to the coordinates of context vector representations during the process of training. Results show an 8{\%} improvement in accuracy on the challenging Bigger Analogy Test Set, and smaller but consistent improvements on other analogy reference sets. These findings demonstrate the importance of order-based information in analogical retrieval tasks, and the utility of random permutations as a means to augment neural embeddings.
Aggregated Semantic Matching for Short Text Entity Linking
Authors: Feng Nie, Shuyan Zhou, Jing Liu, Jinpeng Wang, Chin-Yew Lin, Rong pan
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Card Games, Entity Linking, Information Retrieval, Named Entity Recognition (NER)

The task of entity linking aims to identify concepts mentioned in a text fragments and link them to a reference knowledge base. Entity linking in long text has been well studied in previous work. However, short text entity linking is more challenging since the text are noisy and less coherent. To better utilize the local information provided in short texts, we propose a novel neural network framework, Aggregated Semantic Matching (ASM), in which two different aspects of semantic information between the local context and the candidate entity are captured via representation-based and interaction-based neural semantic matching models, and then two matching signals work jointly for disambiguation with a rank aggregation mechanism. Our evaluation shows that the proposed model outperforms the state-of-the-arts on public tweet datasets.
Modelling Salient Features as Directions in Fine-Tuned Semantic Spaces
Authors: Thomas Ager, Ond{\v{r}}ej Ku{\v{z}}elka, Steven Schockaert
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Knowledge Graph Embeddings, Recommendation Systems, Word Embeddings

In this paper we consider semantic spaces consisting of objects from some particular domain (e.g. IMDB movie reviews). Various authors have observed that such semantic spaces often model salient features (e.g. how scary a movie is) as directions. These feature directions allow us to rank objects according to how much they have the corresponding feature, and can thus play an important role in interpretable classifiers, recommendation systems, or entity-oriented search engines, among others. Methods for learning semantic spaces, however, are mostly aimed at modelling similarity. In this paper, we argue that there is an inherent trade-off between capturing similarity and faithfully modelling features as directions. Following this observation, we propose a simple method to fine-tune existing semantic spaces, with the aim of improving the quality of their feature directions. Crucially, our method is fully unsupervised, requiring only a bag-of-words representation of the objects as input.
Model Transfer with Explicit Knowledge of the Relation between Class Definitions
Authors: Hiyori Yoshikawa, Tomoya Iwakura
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Classification, General Classification, Multi-class Classification, Multi-Task Learning, Named Entity Recognition (NER), Relation, Text Classification

This paper investigates learning methods for multi-class classification using labeled data for the target classification scheme and another labeled data for a similar but different classification scheme (support scheme). We show that if we have prior knowledge about the relation between support and target classification schemes in the form of a class correspondence table, we can use it to improve the model performance further than the simple multi-task learning approach. Instead of learning the individual classification layers for the support and target schemes, the proposed method converts the class label of each example on the support scheme into a set of candidate class labels on the target scheme via the class correspondence table, and then uses the candidate labels to learn the classification layer for the target scheme. We evaluate the proposed method on two tasks in NLP. The experimental results show that our method effectively learns the target schemes especially for the classes that have a tight connection to certain support classes.
Neural Maximum Subgraph Parsing for Cross-Domain Semantic Dependency Analysis
Authors: Yufei Chen, Sheng Huang, Fang Wang, Junjie Cao, Weiwei Sun, Xiaojun Wan
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Semantic Parsing

We present experiments for cross-domain semantic dependency analysis with a neural Maximum Subgraph parser. Our parser targets 1-endpoint-crossing, pagenumber-2 graphs which are a good fit to semantic dependency graphs, and utilizes an efficient dynamic programming algorithm for decoding. For disambiguation, the parser associates words with BiLSTM vectors and utilizes these vectors to assign scores to candidate dependencies. We conduct experiments on the data sets from SemEval 2015 as well as Chinese CCGBank. Our parser achieves very competitive results for both English and Chinese. To improve the parsing performance on cross-domain texts, we propose a data-oriented method to explore the linguistic generality encoded in English Resource Grammar, which is a precisionoriented, hand-crafted HPSG grammar, in an implicit way. Experiments demonstrate the effectiveness of our data-oriented method across a wide range of conditions.
CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies
Authors: Daniel Zeman, Jan Haji{\v{c}}, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, Slav Petrov
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Morphological Analysis, Part-Of-Speech Tagging

Every year, the Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2018, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on test input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. This shared task constitutes a 2nd edition{---}the first one took place in 2017 (Zeman et al., 2017); the main metric from 2017 has been kept, allowing for easy comparison, also in 2018, and two new main metrics have been used. New datasets added to the Universal Dependencies collection between mid-2017 and the spring of 2018 have contributed to increased difficulty of the task this year. In this overview paper, we define the task and the updated evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.
The 2018 Shared Task on Extrinsic Parser Evaluation: On the Downstream Utility of English Universal Dependency Parsers
Authors: Murhaf Fares, Stephan Oepen, Lilja {\O}vrelid, Jari Bj{\"o}rne, Richard Johansson
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Fine-Grained Opinion Analysis, MORPH

We summarize empirical results and tentative conclusions from the Second Extrinsic Parser Evaluation Initiative (EPE 2018). We review the basic task setup, downstream applications involved, and end-to-end results for seventeen participating teams. Based on in-depth quantitative and qualitative analysis, we correlate intrinsic evaluation results at different layers of morph-syntactic analysis with observed downstream behavior.
CEA LIST: Processing Low-Resource Languages for CoNLL 2018
Authors: Elie Duthoo, Olivier Mesnard
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Word Alignment, Word Embeddings

In this paper, we describe the system used for our first participation at the CoNLL 2018 shared task. The submitted system largely reused the state of the art parser from CoNLL 2017 (\url{https://github.com/tdozat/Parser-v2}). We enhanced this system for morphological features predictions, and we used all available resources to provide accurate models for low-resource languages. We ranked 5th of 27 participants in MLAS for building morphology aware dependency trees, 2nd for morphological features only, and 3rd for tagging (UPOS) and parsing (LAS) low-resource languages.
Joint Learning of POS and Dependencies for Multilingual Universal Dependency Parsing
Authors: Zuchao Li, Shexia He, Zhuosheng Zhang, Hai Zhao
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Lemmatization, Part-Of-Speech Tagging, POS, TAG, Transition-Based Dependency Parsing

This paper describes the system of team LeisureX in the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Our system predicts the part-of-speech tag and dependency tree jointly. For the basic tasks, including tokenization, lemmatization and morphology prediction, we employ the official baseline model (UDPipe). To train the low-resource languages, we adopt a sampling method based on other richresource languages. Our system achieves a macro-average of 68.31{\%} LAS F1 score, with an improvement of 2.51{\%} compared with the UDPipe.
Multilingual Universal Dependency Parsing from Raw Text with Low-Resource Language Enhancement
Authors: Yingting Wu, Hai Zhao, Jia-Jun Tong
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Part-Of-Speech Tagging

This paper describes the system of our team Phoenix for participating CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Given the annotated gold standard data in CoNLL-U format, we train the tokenizer, tagger and parser separately for each treebank based on an open source pipeline tool UDPipe. Our system reads the plain texts for input, performs the pre-processing steps (tokenization, lemmas, morphology) and finally outputs the syntactic dependencies. For the low-resource languages with no training data, we use cross-lingual techniques to build models with some close languages instead. In the official evaluation, our system achieves the macro-averaged scores of 65.61{\%}, 52.26{\%}, 55.71{\%} for LAS, MLAS and BLEX respectively.
IBM Research at the CoNLL 2018 Shared Task on Multilingual Parsing
Authors: Hui Wan, Tahira Naseem, Young-suk Lee, Vittorio Castelli, Miguel Ballesteros
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: ARC, Dependency Parsing, Morphological Tagging, Part-Of-Speech Tagging, Sentence, Sentence segmentation

This paper presents the IBM Research AI submission to the CoNLL 2018 Shared Task on Parsing Universal Dependencies. Our system implements a new joint transition-based parser, based on the Stack-LSTM framework and the Arc-Standard algorithm, that handles tokenization, part-of-speech tagging, morphological tagging and dependency parsing in one single model. By leveraging a combination of character-based modeling of words and recursive composition of partially built linguistic structures we qualified 13th overall and 7th in low resource. We also present a new sentence segmentation neural architecture based on Stack-LSTMs that was the 4th best overall.
Tree-Stack LSTM in Transition Based Dependency Parsing
Authors: {\"O}mer K{\i}rnap, Erenay Dayan{\i}k, Deniz Yuret
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Lemmatization, Morphological Analysis, Morphological Tagging, Transition-Based Dependency Parsing, Word Embeddings

We introduce tree-stack LSTM to model state of a transition based parser with recurrent neural networks. Tree-stack LSTM does not use any parse tree based or hand-crafted features, yet performs better than models with these features. We also develop new set of embeddings from raw features to enhance the performance. There are 4 main components of this model: stack{'}s σ-LSTM, buffer{'}s β-LSTM, actions{'} LSTM and tree-RNN. All LSTMs use continuous dense feature vectors (embeddings) as an input. Tree-RNN updates these embeddings based on transitions. We show that our model improves performance with low resource languages compared with its predecessors. We participate in CoNLL 2018 UD Shared Task as the {``}KParse{''} team and ranked 16th in LAS, 15th in BLAS and BLEX metrics, of 27 participants parsing 82 test sets from 57 languages.
Turku Neural Parser Pipeline: An End-to-End System for the CoNLL 2018 Shared Task
Authors: Jenna Kanerva, Filip Ginter, Niko Miekka, Akseli Leino, Tapio Salakoski
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Lemmatization, Machine Translation, Morphological Tagging, Word Embeddings

In this paper we describe the TurkuNLP entry at the CoNLL 2018 Shared Task on Multilingual Parsing from Raw Text to Universal Dependencies. Compared to the last year, this year the shared task includes two new main metrics to measure the morphological tagging and lemmatization accuracies in addition to syntactic trees. Basing our motivation into these new metrics, we developed an end-to-end parsing pipeline especially focusing on developing a novel and state-of-the-art component for lemmatization. Our system reached the highest aggregate ranking on three main metrics out of 26 teams by achieving 1st place on metric involving lemmatization, and 2nd on both morphological tagging and parsing.
SEx BiST: A Multi-Source Trainable Parser with Deep Contextualized Lexical Representations
Authors: KyungTae Lim, Cheoneum Park, Changki Lee, Thierry Poibeau
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, Event Extraction

We describe the SEx BiST parser (Semantically EXtended Bi-LSTM parser) developed at Lattice for the CoNLL 2018 Shared Task (Multilingual Parsing from Raw Text to Universal Dependencies). The main characteristic of our work is the encoding of three different modes of contextual information for parsing: (i) Treebank feature representations, (ii) Multilingual word representations, (iii) ELMo representations obtained via unsupervised learning from external resources. Our parser performed well in the official end-to-end evaluation (73.02 LAS {--} 4th/26 teams, and 78.72 UAS {--} 2nd/26); remarkably, we achieved the best UAS scores on all the English corpora by applying the three suggested feature representations. Finally, we were also ranked 1st at the optional event extraction task, part of the 2018 Extrinsic Parser Evaluation campaign.
The SLT-Interactions Parsing System at the CoNLL 2018 Shared Task
Authors: Riyaz A. Bhat, Irshad Bhat, Srinivas Bangalore
Date: 2018-10-01 00:00:00
Proceeding: CONLL 2018 10
Tasks: ARC, Dependency Parsing, Part-Of-Speech Tagging, POS, POS Tagging, Segmentation, Transfer Learning, Transliteration

This paper describes our system (SLT-Interactions) for the CoNLL 2018 shared task: Multilingual Parsing from Raw Text to Universal Dependencies. Our system performs three main tasks: word segmentation (only for few treebanks), POS tagging and parsing. While segmentation is learned separately, we use neural stacking for joint learning of POS tagging and parsing tasks. For all the tasks, we employ simple neural network architectures that rely on long short-term memory (LSTM) networks for learning task-dependent features. At the basis of our parser, we use an arc-standard algorithm with Swap action for general non-projective parsing. Additionally, we use neural stacking as a knowledge transfer mechanism for cross-domain parsing of low resource domains. Our system shows substantial gains against the UDPipe baseline, with an average improvement of 4.18{\%} in LAS across all languages. Overall, we are placed at the 12th position on the official test sets.