pwc-archive/papers-with-abstracts huggingface dataset:
Learning Contextual Embeddings for Structural Semantic Similarity using Categorical Information
Authors: Massimo Nicosia, Aless Moschitti, ro
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Feature Engineering, Question Answering, Relation Extraction, Semantic Similarity, Semantic Textual Similarity, Sentiment Analysis, Sentiment Classification, text similarity, Word Similarity

Tree kernels (TKs) and neural networks are two effective approaches for automatic feature engineering. In this paper, we combine them by modeling context word similarity in semantic TKs. This way, the latter can operate subtree matching by applying neural-based similarity on tree lexical nodes. We study how to learn representations for the words in context such that TKs can exploit more focused information. We found that neural embeddings produced by current methods do not provide a suitable contextual similarity. Thus, we define a new approach based on a Siamese Network, which produces word representations while learning a binary text similarity. We set the latter considering examples in the same category as similar. The experiments on question and sentiment classification show that our semantic TK highly improves previous results.
A phoneme clustering algorithm based on the obligatory contour principle
Authors: Mans Hulden
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Clustering

This paper explores a divisive hierarchical clustering algorithm based on the well-known Obligatory Contour Principle in phonology. The purpose is twofold: to see if such an algorithm could be used for unsupervised classification of phonemes or graphemes in corpora, and to investigate whether this purported universal constraint really holds for several classes of phonological distinctive features. The algorithm achieves very high accuracies in an unsupervised setting of inferring a consonant-vowel distinction, and also has a strong tendency to detect coronal phonemes in an unsupervised fashion. Remaining classes, however, do not correspond as neatly to phonological distinctive feature splits. While the results offer only mixed support for a universal Obligatory Contour Principle, the algorithm can be very useful for many NLP tasks due to the high accuracy in revealing consonant/vowel/coronal distinctions.
Learning Stock Market Sentiment Lexicon and Sentiment-Oriented Word Vector from StockTwits
Authors: Quanzhi Li, Sameena Shah
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Decision Making, Sentiment Analysis, Word Embeddings

Previous studies have shown that investor sentiment indicators can predict stock market change. A domain-specific sentiment lexicon and sentiment-oriented word embedding model would help the sentiment analysis in financial domain and stock market. In this paper, we present a new approach to learning stock market lexicon from StockTwits, a popular financial social network for investors to share ideas. It learns word polarity by predicting message sentiment, using a neural net-work. The sentiment-oriented word embeddings are learned from tens of millions of StockTwits posts, and this is the first study presenting sentiment-oriented word embeddings for stock market. The experiments of predicting investor sentiment show that our lexicon outperformed other lexicons built by the state-of-the-art methods, and the sentiment-oriented word vector was much better than the general word embeddings.
The Covert Helps Parse the Overt
Authors: Xun Zhang, Weiwei Sun, Xiaojun Wan
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing

This paper is concerned with whether deep syntactic information can help surface parsing, with a particular focus on empty categories. We design new algorithms to produce dependency trees in which empty elements are allowed, and evaluate the impact of information about empty category on parsing overt elements. Such information is helpful to reduce the approximation error in a structured parsing model, but increases the search space for inference and accordingly the estimation error. To deal with structure-based overfitting, we propose to integrate disambiguation models with and without empty elements, and perform structure regularization via joint decoding. Experiments on English and Chinese TreeBanks with different parsing models indicate that incorporating empty elements consistently improves surface parsing.
Multilingual Semantic Parsing And Code-Switching
Authors: Long Duong, Hadi Afshar, Dominique Estival, Glen Pink, Philip Cohen, Mark Johnson
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Domain Adaptation, Semantic Parsing, Transfer Learning, Word Embeddings

Extending semantic parsing systems to new domains and languages is a highly expensive, time-consuming process, so making effective use of existing resources is critical. In this paper, we describe a transfer learning method using crosslingual word embeddings in a sequence-to-sequence model. On the NLmaps corpus, our approach achieves state-of-the-art accuracy of 85.7{\%} for English. Most importantly, we observed a consistent improvement for German compared with several baseline domain adaptation techniques. As a by-product of this approach, our models that are trained on a combination of English and German utterances perform reasonably well on code-switching utterances which contain a mixture of English and German, even though the training data does not contain any such. As far as we know, this is the first study of code-switching in semantic parsing. We manually constructed the set of code-switching test utterances for the NLmaps corpus and achieve 78.3{\%} accuracy on this dataset.
Joint Prediction of Morphosyntactic Categories for Fine-Grained Arabic Part-of-Speech Tagging Exploiting Tag Dictionary Information
Authors: Go Inoue, Hiroyuki Shindo, Yuji Matsumoto
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Multi-Task Learning, Part-Of-Speech Tagging, POS, POS Tagging, TAG, Transliteration

Part-of-speech (POS) tagging for morphologically rich languages such as Arabic is a challenging problem because of their enormous tag sets. One reason for this is that in the tagging scheme for such languages, a complete POS tag is formed by combining tags from multiple tag sets defined for each morphosyntactic category. Previous approaches in Arabic POS tagging applied one model for each morphosyntactic tagging task, without utilizing shared information between the tasks. In this paper, we propose an approach that utilizes this information by jointly modeling multiple morphosyntactic tagging tasks with a multi-task learning framework. We also propose a method of incorporating tag dictionary information into our neural models by combining word representations with representations of the sets of possible tags. Our experiments showed that the joint model with tag dictionary information results in an accuracy of 91.38{\%} on the Penn Arabic Treebank data set, with an absolute improvement of 2.11{\%} over the current state-of-the-art tagger.
Learning from Relatives: Unified Dialectal Arabic Segmentation
Authors: Younes Samih, Mohamed Eldesouki, Mohammed Attia, Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, Laura Kallmeyer
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dialect Identification, Information Retrieval, Machine Translation, Segmentation

Arabic dialects do not just share a common koin{\'e}, but there are shared pan-dialectal linguistic phenomena that allow computational models for dialects to learn from each other. In this paper we build a unified segmentation model where the training data for different dialects are combined and a single model is trained. The model yields higher accuracies than dialect-specific models, eliminating the need for dialect identification before segmentation. We also measure the degree of relatedness between four major Arabic dialects by testing how a segmentation model trained on one dialect performs on the other dialects. We found that linguistic relatedness is contingent with geographical proximity. In our experiments we use SVM-based ranking and bi-LSTM-CRF sequence labeling.
CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies
Authors: Daniel Zeman, Martin Popel, Milan Straka, Jan Haji{\v{c}}, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gokirmak, Anna Nedoluzhko, Silvie Cinkov{\'a}, Jan Haji{\v{c}} jr., Jaroslava Hlav{\'a}{\v{c}}ov{\'a}, V{\'a}clava Kettnerov{\'a}, Zde{\v{n}}ka Ure{\v{s}}ov{\'a}, Jenna Kanerva, Stina Ojala, Anna Missil{\"a}, Christopher D. Manning, Sebastian Schuster, Siva Reddy, Dima Taji, Nizar Habash, Herman Leung, Marie-Catherine de Marneffe, Manuela Sanguinetti, Maria Simi, Hiroshi Kanayama, Valeria de Paiva, Kira Droganova, H{\'e}ctor Mart{\'\i}nez Alonso, {\c{C}}a{\u{g}}r{\i} {\c{C}}{\"o}ltekin, Umut Sulubacak, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Georg Rehm, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, M, Michael l, Jesse Kirchner, Hector Fern Alcalde, ez, Jana Strnadov{\'a}, Esha Banerjee, Ruli Manurung, Antonio Stella, Atsuko Shimada, Sookyoung Kwak, Gustavo Mendon{\c{c}}a, L, Tatiana o, Rattima Nitisaroj, Josie Li
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing

The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, the task was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe how the data sets were prepared, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.
Stanford's Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task
Authors: Timothy Dozat, Peng Qi, Christopher D. Manning
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing

This paper describes the neural dependency parser submitted by Stanford to the CoNLL 2017 Shared Task on parsing Universal Dependencies. Our system uses relatively simple LSTM networks to produce part of speech tags and labeled dependency parses from segmented and tokenized sequences of words. In order to address the rare word problem that abounds in languages with complex morphology, we include a character-based word representation that uses an LSTM to produce embeddings from sequences of characters. Our system was ranked first according to all five relevant metrics for the system: UPOS tagging (93.09{\%}), XPOS tagging (82.27{\%}), unlabeled attachment score (81.30{\%}), labeled attachment score (76.30{\%}), and content word labeled attachment score (72.57{\%}).
Combining Global Models for Parsing Universal Dependencies
Authors: Tianze Shi, Felix G. Wu, Xilun Chen, Yao Cheng
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Boundary Detection, Dependency Parsing, Part-Of-Speech Tagging

We describe our entry, C2L2, to the CoNLL 2017 shared task on parsing Universal Dependencies from raw text. Our system features an ensemble of three global parsing paradigms, one graph-based and two transition-based. Each model leverages character-level bi-directional LSTMs as lexical feature extractors to encode morphological information. Though relying on baseline tokenizers and focusing only on parsing, our system ranked second in the official end-to-end evaluation with a macro-average of 75.00 LAS F1 score over 81 test treebanks. In addition, we had the top average performance on the four surprise languages and on the small treebank subset.
IMS at the CoNLL 2017 UD Shared Task: CRFs and Perceptrons Meet Neural Networks
Authors: Anders Bj{\"o}rkelund, Agnieszka Falenska, Xiang Yu, Jonas Kuhn
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: POS, Segmentation, Sentence, Sentence segmentation

This paper presents the IMS contribution to the CoNLL 2017 Shared Task. In the preprocessing step we employed a CRF POS/morphological tagger and a neural tagger predicting supertags. On some languages, we also applied word segmentation with the CRF tagger and sentence segmentation with a perceptron-based parser. For parsing we took an ensemble approach by blending multiple instances of three parsers with very different architectures. Our system achieved the third place overall and the second place for the surprise languages.
The HIT-SCIR System for End-to-End Parsing of Universal Dependencies
Authors: Wanxiang Che, Jiang Guo, Yuxuan Wang, Bo Zheng, Huaipeng Zhao, Yang Liu, Dechuan Teng, Ting Liu
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Information Retrieval, Part-Of-Speech Tagging, POS, POS Tagging, Transfer Learning

This paper describes our system (HIT-SCIR) for the CoNLL 2017 shared task: Multilingual Parsing from Raw Text to Universal Dependencies. Our system includes three pipelined components: \textit{tokenization}, \textit{Part-of-Speech} (POS) \textit{tagging} and \textit{dependency parsing}. We use character-based bidirectional long short-term memory (LSTM) networks for both tokenization and POS tagging. Afterwards, we employ a list-based transition-based algorithm for general non-projective parsing and present an improved Stack-LSTM-based architecture for representing each transition state and making predictions. Furthermore, to parse low/zero-resource languages and cross-domain data, we use a model transfer approach to make effective use of existing resources. We demonstrate substantial gains against the UDPipe baseline, with an average improvement of 3.76{\%} in LAS of all languages. And finally, we rank the 4th place on the official test sets.
A System for Multilingual Dependency Parsing based on Bidirectional LSTM Feature Representations
Authors: KyungTae Lim, Thierry Poibeau
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Multilingual Word Embeddings, Word Embeddings

In this paper, we present our multilingual dependency parser developed for the CoNLL 2017 UD Shared Task dealing with {``}Multilingual Parsing from Raw Text to Universal Dependencies{''}. Our parser extends the monolingual BIST-parser as a multi-source multilingual trainable parser. Thanks to multilingual word embeddings and one hot encodings for languages, our system can use both monolingual and multi-source training. We trained 69 monolingual language models and 13 multilingual models for the shared task. Our multilingual approach making use of different resources yield better results than the monolingual approach for 11 languages. Our system ranked 5 th and achieved 70.93 overall LAS score over the 81 test corpora (macro-averaged LAS F1 score).
Adversarial Training for Cross-Domain Universal Dependency Parsing
Authors: Motoki Sato, Hitoshi Manabe, Hiroshi Noji, Yuji Matsumoto
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Domain Adaptation

We describe our submission to the CoNLL 2017 shared task, which exploits the shared common knowledge of a language across different domains via a domain adaptation technique. Our approach is an extension to the recently proposed adversarial training technique for domain adaptation, which we apply on top of a graph-based neural dependency parsing model on bidirectional LSTMs. In our experiments, we find our baseline graph-based parser already outperforms the official baseline model (UDPipe) by a large margin. Further, by applying our technique to the treebanks of the same language with different domains, we observe an additional gain in the performance, in particular for the domains with less training data.
Parsing with Context Embeddings
Authors: {\"O}mer K{\i}rnap, Berkay Furkan {\"O}nder, Deniz Yuret
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Language Modeling, Language Modelling, Word Embeddings, Word Sense Induction

We introduce context embeddings, dense vectors derived from a language model that represent the left/right context of a word instance, and demonstrate that context embeddings significantly improve the accuracy of our transition based parser. Our model consists of a bidirectional LSTM (BiLSTM) based language model that is pre-trained to predict words in plain text, and a multi-layer perceptron (MLP) decision model that uses features from the language model to predict the correct actions for an ArcHybrid transition based parser. We participated in the CoNLL 2017 UD Shared Task as the {``}Ko{\c{c}} University{''} team and our system was ranked 7th out of 33 systems that parsed 81 treebanks in 49 languages.
Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe
Authors: Milan Straka, Jana Strakov{\'a}
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Lemmatization, POS, POS Tagging

Many natural language processing tasks, including the most advanced ones, routinely start by several basic processing steps {--} tokenization and segmentation, most likely also POS tagging and lemmatization, and commonly parsing as well. A multilingual pipeline performing these steps can be trained using the Universal Dependencies project, which contains annotations of the described tasks for 50 languages in the latest release UD 2.0. We present an update to UDPipe, a simple-to-use pipeline processing CoNLL-U version 2.0 files, which performs these tasks for multiple languages without requiring additional external data. We provide models for all 50 languages of UD 2.0, and furthermore, the pipeline can be trained easily using data in CoNLL-U format. UDPipe is a standalone application in C++, with bindings available for Python, Java, C{\#} and Perl. In the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, UDPipe was the eight best system, while achieving low running times and moderately sized models.
UParse: the Edinburgh system for the CoNLL 2017 UD shared task
Authors: Clara Vania, Xingxing Zhang, Adam Lopez
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Machine Translation, Question Answering, Sentence, Transfer Learning

This paper presents our submissions for the CoNLL 2017 UD Shared Task. Our parser, called UParse, is based on a neural network graph-based dependency parser. The parser uses features from a bidirectional LSTM to to produce a distribution over possible heads for each word in the sentence. To allow transfer learning for low-resource treebanks and surprise languages, we train several multilingual models for related languages, grouped by their genus and language families. Out of 33 participants, our system achieves rank 9th in the main results, with 75.49 UAS and 68.87 LAS F-1 scores (average across 81 treebanks).
Multi-Model and Crosslingual Dependency Analysis
Authors: Johannes Heinecke, Munshi Asadullah
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, model, Word Embeddings

This paper describes the system of the Team Orange-Deski{\~n}, used for the CoNLL 2017 UD Shared Task in Multilingual Dependency Parsing. We based our approach on an existing open source tool (BistParser), which we modified in order to produce the required output. Additionally we added a kind of pseudo-projectivisation. This was needed since some of the task{'}s languages have a high percentage of non-projective dependency trees. In most cases we also employed word embeddings. For the 4 surprise languages, the data provided seemed too little to train on. Thus we decided to use the training data of typologically close languages instead. Our system achieved a macro-averaged LAS of 68.61{\%} (10th in the overall ranking) which improved to 69.38{\%} after bug fixes.
TurkuNLP: Delexicalized Pre-training of Word Embeddings for Dependency Parsing
Authors: Jenna Kanerva, Juhani Luotolahti, Filip Ginter
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Word Embeddings

We present the TurkuNLP entry in the CoNLL 2017 Shared Task on Multilingual Parsing from Raw Text to Universal Dependencies. The system is based on the UDPipe parser with our focus being in exploring various techniques to pre-train the word embeddings used by the parser in order to improve its performance especially on languages with small training sets. The system ranked 11th among the 33 participants overall, being 8th on the small treebanks, 10th on the large treebanks, 12th on the parallel test sets, and 26th on the surprise languages.
The parse is darc and full of errors: Universal dependency parsing with transition-based and graph-based algorithms
Authors: Kuan Yu, Pavel Sofroniev, Erik Schill, Erhard Hinrichs
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing

We developed two simple systems for dependency parsing: darc, a transition-based parser, and mstnn, a graph-based parser. We tested our systems in the CoNLL 2017 UD Shared Task, with darc being the official system. Darc ranked 12th among 33 systems, just above the baseline. Mstnn had no official ranking, but its main score was above the 27th. In this paper, we describe our two systems, examine their strengths and weaknesses, and discuss the lessons we learned.
A non-DNN Feature Engineering Approach to Dependency Parsing -- FBAML at CoNLL 2017 Shared Task
Authors: Xian Qian, Yang Liu
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Dependency Parsing, Feature Engineering, MORPH, Morphological Analysis, Part-Of-Speech Tagging, Relation Classification, Sentence, Word Embeddings

For this year{'}s multilingual dependency parsing shared task, we developed a pipeline system, which uses a variety of features for each of its components. Unlike the recent popular deep learning approaches that learn low dimensional dense features using non-linear classifier, our system uses structured linear classifiers to learn millions of sparse features. Specifically, we trained a linear classifier for sentence boundary prediction, linear chain conditional random fields (CRFs) for tokenization, part-of-speech tagging and morph analysis. A second order graph based parser learns the tree structure (without relations), and fa linear tree CRF then assigns relations to the dependencies in the tree. Our system achieves reasonable performance {--} 67.87{\%} official averaged macro F1 score
LIMSI@CoNLL'17: UD Shared Task
Authors: Lauriane Aufrant, Guillaume Wisniewski, Fran{\c{c}}ois Yvon
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Model Selection

This paper describes LIMSI{'}s submission to the CoNLL 2017 UD Shared Task, which is focused on small treebanks, and how to improve low-resourced parsing only by ad hoc combination of multiple views and resources. We present our approach for low-resourced parsing, together with a detailed analysis of the results for each test treebank. We also report extensive analysis experiments on model selection for the PUD treebanks, and on annotation consistency among UD treebanks.
RACAI's Natural Language Processing pipeline for Universal Dependencies
Authors: Stefan Daniel Dumitrescu, Tiberiu Boros, Dan Tufis
Date: 2017-08-01 00:00:00
Proceeding: CONLL 2017 8
Tasks: Lemmatization, Sentence

This paper presents RACAI{'}s approach, experiments and results at CONLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. We handle raw text and we cover tokenization, sentence splitting, word segmentation, tagging, lemmatization and parsing. All results are reported under strict training, development and testing conditions, in which the corpora provided for the shared tasks is used {``}as is{''}, without any modifications to the composition of the train and development sets.