pwc-archive/papers-with-abstracts huggingface dataset:
Relations between comprehensibility and adequacy errors in machine translation output
Authors: Maja Popovi{\'c}
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Machine Translation, Translation, Word Translation

This work presents a detailed analysis of translation errors perceived by readers as comprehensibility and/or adequacy issues. The main finding is that good comprehensibility, similarly to good fluency, can mask a number of adequacy errors. Of all major adequacy errors, 30{\%} were fully comprehensible, thus fully misleading the reader to accept the incorrect information. Another 25{\%} of major adequacy errors were perceived as almost comprehensible, thus being potentially misleading. Also, a vast majority of omissions (about 70{\%}) is hidden by comprehensibility. Further analysis of misleading translations revealed that the most frequent error types are ambiguity, mistranslation, noun phrase error, word-by-word translation, untranslated word, subject-verb agreement, and spelling error in the source text. However, none of these error types appears exclusively in misleading translations, but are also frequent in fully incorrect (incomprehensible inadequate) and discarded correct (incomprehensible adequate) translations. Deeper analysis is needed to potentially detect underlying phenomena specifically related to misleading translations.
Cross-lingual Embeddings Reveal Universal and Lineage-Specific Patterns in Grammatical Gender Assignment
Authors: Hartger Veeman, Marc Allassonni{\`e}re-Tang, Aleksandrs Berdicevskis, Ali Basirat
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Word Embeddings

Grammatical gender is assigned to nouns differently in different languages. Are all factors that influence gender assignment idiosyncratic to languages or are there any that are universal? Using cross-lingual aligned word embeddings, we perform two experiments to address these questions about language typology and human cognition. In both experiments, we predict the gender of nouns in language X using a classifier trained on the nouns of language Y, and take the classifier{'}s accuracy as a measure of transferability of gender systems. First, we show that for 22 Indo-European languages the transferability decreases as the phylogenetic distance increases. This correlation supports the claim that some gender assignment factors are idiosyncratic, and as the languages diverge, the proportion of shared inherited idiosyncrasies diminishes. Second, we show that when the classifier is trained on two Afro-Asiatic languages and tested on the same 22 Indo-European languages (or vice versa), its performance is still significantly above the chance baseline, thus showing that universal factors exist and, moreover, can be captured by word embeddings. When the classifier is tested across families and on inanimate nouns only, the performance is still above baseline, indicating that the universal factors are not limited to biological sex.
Learning to ground medical text in a 3D human atlas
Authors: Dusan Grujicic, Gorjan Radevski, Tinne Tuytelaars, Matthew Blaschko
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Phrase Grounding, Visual Grounding

In this paper, we develop a method for grounding medical text into a physically meaningful and interpretable space corresponding to a human atlas. We build on text embedding architectures such as Bert and introduce a loss function that allows us to reason about the semantic and spatial relatedness of medical texts by learning a projection of the embedding into a 3D space representing the human body. We quantitatively and qualitatively demonstrate that our proposed method learns a context sensitive and spatially aware mapping, in both the inter-organ and intra-organ sense, using a large scale medical text dataset from the {``}Large-scale online biomedical semantic indexing{''} track of the 2020 BioASQ challenge. We extend our approach to a self-supervised setting, and find it to be competitive with a classification based method, and a fully supervised variant of approach.
Representation Learning for Type-Driven Composition
Authors: Gijs Wijnholds, Mehrnoosh Sadrzadeh, Stephen Clark
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Representation Learning, Sentence, Sentence Similarity, Vocal Bursts Type Prediction

This paper is about learning word representations using grammatical type information. We use the syntactic types of Combinatory Categorial Grammar to develop multilinear representations, i.e. maps with n arguments, for words with different functional types. The multilinear maps of words compose with each other to form sentence representations. We extend the skipgram algorithm from vectors to multi- linear maps to learn these representations and instantiate it on unary and binary maps for transitive verbs. These are evaluated on verb and sentence similarity and disambiguation tasks and a subset of the SICK relatedness dataset. Our model performs better than previous type- driven models and is competitive with state of the art representation learning methods such as BERT and neural sentence encoders.
Word Representations Concentrate and This is Good News!
Authors: Romain Couillet, Yagmur Gizem Cinar, Eric Gaussier, Muhammad Imran
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

This article establishes that, unlike the legacy tf*idf representation, recent natural language representations (word embedding vectors) tend to exhibit a so-called \textit{concentration of measure phenomenon}, in the sense that, as the representation size $p$ and database size $n$ are both large, their behavior is similar to that of large dimensional Gaussian random vectors. This phenomenon may have important consequences as machine learning algorithms for natural language data could be amenable to improvement, thereby providing new theoretical insights into the field of natural language processing.
``LazImpa'': Lazy and Impatient neural agents learn to communicate efficiently
Authors: Mathieu Rita, Rahma Chaabouni, Emmanuel Dupoux
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

Previous work has shown that artificial neural agents naturally develop surprisingly non-efficient codes. This is illustrated by the fact that in a referential game involving a speaker and a listener neural networks optimizing accurate transmission over a discrete channel, the emergent messages fail to achieve an optimal length. Furthermore, frequent messages tend to be longer than infrequent ones, a pattern contrary to the Zipf Law of Abbreviation (ZLA) observed in all natural languages. Here, we show that near-optimal and ZLA-compatible messages can emerge, but only if both the speaker and the listener are modified. We hence introduce a new communication system, {``}LazImpa{''}, where the speaker is made increasingly lazy, i.e., avoids long messages, and the listener impatient, i.e., seeks to guess the intended content as soon as possible.
Re-solve it: simulating the acquisition of core semantic competences from small data
Authors: Aur{\'e}lie Herbelot
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

Many tasks are considered to be {`}solved{'} in the computational linguistics literature, but the corresponding algorithms operate in ways which are radically different from human cognition. I illustrate this by coming back to the notion of semantic competence, which includes basic linguistic skills encompassing both referential phenomena and generic knowledge, in particular a) the ability to denote, b) the mastery of the lexicon, or c) the ability to model one{'}s language use on others. Even though each of those faculties has been extensively tested individually, there is still no computational model that would account for their joint acquisition under the conditions experienced by a human. In this paper, I focus on one particular aspect of this problem: the amount of linguistic data available to the child or machine. I show that given the first competence mentioned above (a denotation function), the other two can in fact be learned from very limited data (2.8M token), reaching state-of-the-art performance. I argue that both the nature of the data and the way it is presented to the system matter to acquisition.
In Media Res: A Corpus for Evaluating Named Entity Linking with Creative Works
Authors: Adrian M.P. Brasoveanu, Albert Weichselbraun, Lyndon Nixon
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Entity Linking

Annotation styles express guidelines that direct human annotators in what rules to follow when creating gold standard annotations of text corpora. These guidelines not only shape the gold standards they help create, but also influence the training and evaluation of Named Entity Linking (NEL) tools, since different annotation styles correspond to divergent views on the entities present in the same texts. Such divergence is particularly present in texts from the media domain that contain references to creative works. In this work we present a corpus of 1000 annotated documents selected from the media domain. Each document is presented with multiple gold standard annotations representing various annotation styles. This corpus is used to evaluate a series of Named Entity Linking tools in order to understand the impact of the differences in annotation styles on the reported accuracy when processing highly ambiguous entities such as names of creative works. Relaxed annotation guidelines that include overlap styles lead to better results across all tools.
Word associations and the distance properties of context-aware word embeddings
Authors: Maria A. Rodriguez, Paola Merlo
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Word Embeddings

What do people know when they know the meaning of words? Word associations have been widely used to tap into lexical repre- sentations and their structure, as a way of probing semantic knowledge in humans. We investigate whether current word embedding spaces (contextualized and uncontextualized) can be considered good models of human lexi- cal knowledge by studying whether they have comparable characteristics to human associa- tion spaces. We study the three properties of association rank, asymmetry of similarity and triangle inequality. We find that word embeddings are good mod- els of some word associations properties. They replicate well human associations between words, and, like humans, their context-aware variants show violations of the triangle in- equality. While they do show asymmetry of similarities, their asymmetries do not map those of human association norms.
TrClaim-19: The First Collection for Turkish Check-Worthy Claim Detection with Annotator Rationales
Authors: Yavuz Selim Kartal, Mucahid Kutlu
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Misinformation

Massive misinformation spread over Internet has many negative impacts on our lives. While spreading a claim is easy, investigating its veracity is hard and time consuming, Therefore, we urgently need systems to help human fact-checkers. However, available data resources to develop effective systems are limited and the vast majority of them is for English. In this work, we introduce TrClaim-19, which is the very first labeled dataset for Turkish check-worthy claims. TrClaim-19 consists of labeled 2287 Turkish tweets with annotator rationales, enabling us to better understand the characteristics of check-worthy claims. The rationales we collected suggest that claims{'} topics and their possible negative impacts are the main factors affecting their check-worthiness.
Diverse and Relevant Visual Storytelling with Scene Graph Embeddings
Authors: Xudong Hong, Rakshith Shetty, Asad Sayeed, Khushboo Mehra, Vera Demberg, Bernt Schiele
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Diversity, Story Generation, Visual Storytelling

A problem in automatically generated stories for image sequences is that they use overly generic vocabulary and phrase structure and fail to match the distributional characteristics of human-generated text. We address this problem by introducing explicit representations for objects and their relations by extracting scene graphs from the images. Utilizing an embedding of this scene graph enables our model to more explicitly reason over objects and their relations during story generation, compared to the global features from an object classifier used in previous work. We apply metrics that account for the diversity of words and phrases of generated stories as well as for reference to narratively-salient image features and show that our approach outperforms previous systems. Our experiments also indicate that our models obtain competitive results on reference-based metrics.
Alleviating Digitization Errors in Named Entity Recognition for Historical Documents
Authors: Emanuela Boros, Ahmed Hamdi, Elvys Linhares Pontes, Luis Adri{\'a}n Cabrera-Diego, Jose G. Moreno, Nicolas Sidere, Antoine Doucet
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Optical Character Recognition, Optical Character Recognition (OCR)

This paper tackles the task of named entity recognition (NER) applied to digitized historical texts obtained from processing digital images of newspapers using optical character recognition (OCR) techniques. We argue that the main challenge for this task is that the OCR process leads to misspellings and linguistic errors in the output text. Moreover, historical variations can be present in aged documents, which can impact the performance of the NER process. We conduct a comparative evaluation on two historical datasets in German and French against previous state-of-the-art models, and we propose a model based on a hierarchical stack of Transformers to approach the NER task for historical data. Our findings show that the proposed model clearly improves the results on both historical datasets, and does not degrade the results for modern datasets.
Analysing Word Representation from the Input and Output Embeddings in Neural Network Language Models
Authors: Steven Derby, Paul Miller, Barry Devereux
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Language Modeling, Language Modelling, Word Embeddings, Word Similarity

Researchers have recently demonstrated that tying the neural weights between the input look-up table and the output classification layer can improve training and lower perplexity on sequence learning tasks such as language modelling. Such a procedure is possible due to the design of the softmax classification layer, which previous work has shown to comprise a viable set of semantic representations for the model vocabulary, and these these output embeddings are known to perform well on word similarity benchmarks. In this paper, we make meaningful comparisons between the input and output embeddings and other SOTA distributional models to gain a better understanding of the types of information they represent. We also construct a new set of word embeddings using the output embeddings to create locally-optimal approximations for the intermediate representations from the language model. These locally-optimal embeddings demonstrate excellent performance across all our evaluations.
An Expectation Maximisation Algorithm for Automated Cognate Detection
Authors: Roddy MacSween, Andrew Caines
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

In historical linguistics, cognate detection is the task of determining whether sets of words have common etymological roots. Inspired by the comparative method used by human linguists, we develop a system for automated cognate detection that frames the task as an inference problem for a general statistical model consisting of observed data (potentially cognate pairs of words), latent variables (the cognacy status of pairs) and unknown global parameters (which sounds correspond between languages). We then give a specific instance of such a model along with an expectation-maximisation algorithm to infer its parameters. We evaluate our system on a corpus of 8140 cognate sets, finding the performance of our method to be comparable to the state of the art. We additionally carry out qualitative analysis demonstrating advantages it has over existing systems. We also suggest several ways our work could be extended within the general theoretical framework we propose.
Filler-gaps that neural networks fail to generalize
Authors: Debasmita Bhattacharya, Marten Van Schijndel
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

It can be difficult to separate abstract linguistic knowledge in recurrent neural networks (RNNs) from surface heuristics. In this work, we probe for highly abstract syntactic constraints that have been claimed to govern the behavior of filler-gap dependencies across different surface constructions. For models to generalize abstract patterns in expected ways to unseen data, they must share representational features in predictable ways. We use cumulative priming to test for representational overlap between disparate filler-gap constructions in English and find evidence that the models learn a general representation for the existence of filler-gap dependencies. However, we find no evidence that the models learn any of the shared underlying grammatical constraints we tested. Our work raises questions about the degree to which RNN language models learn abstract linguistic representations.
What Are You Trying to Do? Semantic Typing of Event Processes
Authors: Muhao Chen, Hongming Zhang, Haoyu Wang, Dan Roth
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Learning-To-Rank, Vocal Bursts Type Prediction

This paper studies a new cognitively motivated semantic typing task,multi-axis event process typing, that, given anevent process, attempts to infer free-form typelabels describing (i) the type of action made bythe process and (ii) the type of object the pro-cess seeks to affect. This task is inspired bycomputational and cognitive studies of eventunderstanding, which suggest that understand-ing processes of events is often directed by rec-ognizing the goals, plans or intentions of theprotagonist(s). We develop a large dataset con-taining over 60k event processes, featuring ul-tra fine-grained typing on both the action andobject type axes with very large (10{\^{}}3∼10{\^{}}4)label vocabularies. We then propose a hybridlearning framework,P2GT, which addressesthe challenging typing problem with indirectsupervision from glosses1and a joint learning-to-rank framework. As our experiments indi-cate,P2GTsupports identifying the intent ofprocesses, as well as the fine semantic type ofthe affected object. It also demonstrates the ca-pability of handling few-shot cases, and stronggeneralizability on out-of-domain processes.
A Corpus for Outbreak Detection of Diseases Prevalent in Latin America
Authors: Antonella Dellanzo, Viviana Cotik, Jose Ochoa-Luna
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

In this paper we present an annotated corpus which can be used for training and testing algorithms to automatically extract information about diseases outbreaks from news and health reports. We also propose initial approaches to extract information from it. The corpus has been constructed with two main tasks in mind. The first one, to extract entities about outbreaks such as disease, host, location among others. The second one, to retrieve relations among entities, for instance, in such geographic location fifteen cases of a given disease were reported. Overall, our goal is to offer resources and tools to perform an automated analysis so as to support early detection of disease outbreaks and therefore diminish their spreading.
Understanding Linguistic Accommodation in Code-Switched Human-Machine Dialogues
Authors: Tanmay Parekh, Emily Ahn, Yulia Tsvetkov, Alan W Black
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

Code-switching is a ubiquitous phenomenon in multilingual communities. Natural language technologies that wish to communicate like humans must therefore adaptively incorporate code-switching techniques when they are deployed in multilingual settings. To this end, we propose a Hindi-English human-machine dialogue system that elicits code-switching conversations in a controlled setting. It uses different code-switching agent strategies to understand how users respond and accommodate to the agent{'}s language choice. Through this system, we collect and release a new dataset CommonDost, comprising of 439 human-machine multilingual conversations. We adapt pre-defined metrics to discover linguistic accommodation from users to agents. Finally, we compare these dialogues with Spanish-English dialogues collected in a similar setting, and analyze the impact of linguistic and socio-cultural factors on code-switching patterns across the two language pairs.
Identifying robust markers of Parkinson's disease in typing behaviour using a CNN-LSTM network
Authors: Neil Dhir, Mathias Edman, {\'A}lvaro Sanchez Ferro, Tom Stafford, Colin Bannard
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

There is urgent need for non-intrusive tests that can detect early signs of Parkinson{'}s disease (PD), a debilitating neurodegenerative disorder that affects motor control. Recent promising research has focused on disease markers evident in the fine-motor behaviour of typing. Most work to date has focused solely on the timing of keypresses without reference to the linguistic content. In this paper we argue that the identity of the key combinations being produced should impact how they are handled by people with PD, and provide evidence that natural language processing methods can thus be of help in identifying signs of disease. We test the performance of a bi-directional LSTM with convolutional features in distinguishing people with PD from age-matched controls typing in English and Spanish, both in clinics and online.
Cloze Distillation: Improving Neural Language Models with Human Next-Word Prediction
Authors: Tiwalayo Eisape, Noga Zaslavsky, Roger Levy
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020

Contemporary autoregressive language models (LMs) trained purely on corpus data have been shown to capture numerous features of human incremental processing. However, past work has also suggested dissociations between corpus probabilities and human next-word predictions. Here we evaluate several state-of-the-art language models for their match to human next-word predictions and to reading time behavior from eye movements. We then propose a novel method for distilling the linguistic information implicit in human linguistic predictions into pre-trained LMs: Cloze Distillation. We apply this method to a baseline neural LM and show potential improvement in reading time prediction and generalization to held-out human cloze data.
Disentangling dialects: a neural approach to Indo-Aryan historical phonology and subgrouping
Authors: Chundra Cathcart, Taraka Rama
Date: 2020-11-01 00:00:00
Proceeding: CONLL 2020
Tasks: Decoder, Word Embeddings

This paper seeks to uncover patterns of sound change across Indo-Aryan languages using an LSTM encoder-decoder architecture. We augment our models with embeddings represent-ing language ID, part of speech, and other features such as word embeddings. We find that a highly augmented model shows highest accuracy in predicting held-out forms, and investigate other properties of interest learned by our models{'} representations. We outline extensions to this architecture that can better capture variation in Indo-Aryan sound change.