pwc-archive/papers-with-abstracts huggingface dataset:
RST-Tace A tool for automatic comparison and evaluation of RST trees
Authors: Shujun Wan, Tino Kutschbach, Anke L{\"u}deling, Manfred Stede
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Discourse Parsing

This paper presents RST-Tace, a tool for automatic comparison and evaluation of RST trees. RST-Tace serves as an implementation of Iruskieta{'}s comparison method, which allows trees to be compared and evaluated without the influence of decisions at lower levels in a tree in terms of four factors: constituent, attachment point, nuclearity as well as relation. RST-Tace can be used regardless of the language or the size of rhetorical trees. This tool aims to measure the agreement between two annotators. The result is reflected by F-measure and inter-annotator agreement. Both the comparison table and the result of the evaluation can be obtained automatically.
The DISRPT 2019 Shared Task on Elementary Discourse Unit Segmentation and Connective Detection
Authors: Amir Zeldes, Debopam Das, Erick Galani Maziero, Juliano Antonio, Mikel Iruskieta
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Connective Detection, Discourse Parsing, Survey

In 2019, we organized the first iteration of a shared task dedicated to the underlying units used in discourse parsing across formalisms: the DISRPT Shared Task on Elementary Discourse Unit Segmentation and Connective Detection. In this paper we review the data included in the task, which cover 2.6 million manually annotated tokens from 15 datasets in 10 languages, survey and compare submitted systems and report on system performance on each task for both annotated and plain-tokenized versions of the data.
Multi-lingual and Cross-genre Discourse Unit Segmentation
Authors: Peter Bourgonje, Robin Sch{\"a}fer
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Discourse Segmentation, Segmentation, Sentence

We describe a series of experiments applied to data sets from different languages and genres annotated for coherence relations according to different theoretical frameworks. Specifically, we investigate the feasibility of a unified (theory-neutral) approach toward discourse segmentation; a process which divides a text into minimal discourse units that are involved in s coherence relation. We apply a RandomForest and an LSTM based approach for all data sets, and we improve over a simple baseline assuming simple sentence or clause-like segmentation. Performance however varies a lot depending on language, and more importantly genre, with f-scores ranging from 73.00 to 94.47.
ToNy: Contextual embeddings for accurate multilingual discourse segmentation of full documents
Authors: Philippe Muller, Chlo{\'e} Braud, Mathieu Morey
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Connective Detection, Discourse Parsing, Discourse Segmentation, Segmentation, Sentence

Segmentation is the first step in building practical discourse parsers, and is often neglected in discourse parsing studies. The goal is to identify the minimal spans of text to be linked by discourse relations, or to isolate explicit marking of discourse relations. Existing systems on English report F1 scores as high as 95{\%}, but they generally assume gold sentence boundaries and are restricted to English newswire texts annotated within the RST framework. This article presents a generic approach and a system, ToNy, a discourse segmenter developed for the DisRPT shared task where multiple discourse representation schemes, languages and domains are represented. In our experiments, we found that a straightforward sequence prediction architecture with pretrained contextual embeddings is sufficient to reach performance levels comparable to existing systems, when separately trained on each corpus. We report performance between 81{\%} and 96{\%} in F1 score. We also observed that discourse segmentation models only display a moderate generalization capability, even within the same language and discourse representation scheme.
Multilingual segmentation based on neural networks and pre-trained word embeddings
Authors: Mikel Iruskieta, Kepa Bengoetxea, Aitziber Atutxa Salazar, Arantza Diaz de Ilarraza
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Connective Detection, Discourse Segmentation, Segmentation, Word Embeddings

The DISPRT 2019 workshop has organized a shared task aiming to identify cross-formalism and multilingual discourse segments. Elementary Discourse Units (EDUs) are quite similar across different theories. Segmentation is the very first stage on the way of rhetorical annotation. Still, each annotation project adopted several decisions with consequences not only on the annotation of the relational discourse structure but also at the segmentation stage. In this shared task, we have employed pre-trained word embeddings, neural networks (BiLSTM+CRF) to perform the segmentation. We report F1 results for 6 languages: Basque (0.853), English (0.919), French (0.907), German (0.913), Portuguese (0.926) and Spanish (0.868 and 0.769). Finally, we also pursued an error analysis based on clause typology for Basque and Spanish, in order to understand the performance of the segmenter.
Towards discourse annotation and sentiment analysis of the Basque Opinion Corpus
Authors: Jon Alkorta, Koldo Gojenola, Mikel Iruskieta
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Sentiment Analysis

Discourse information is crucial for a better understanding of the text structure and it is also necessary to describe which part of an opinionated text is more relevant or to decide how a text span can change the polarity (strengthen or weaken) of other span by means of coherence relations. This work presents the first results on the annotation of the Basque Opinion Corpus using Rhetorical Structure Theory (RST). Our evaluation results and analysis show us the main avenues to improve on a future annotation process. We have also extracted the subjectivity of several rhetorical relations and the results show the effect of sentiment words in relations and the influence of each relation in the semantic orientation value.
Using Rhetorical Structure Theory to Assess Discourse Coherence for Non-native Spontaneous Speech
Authors: Xinhao Wang, Binod Gyawali, James V. Bruno, Hillary R. Molloy, Keelan Evanini, Klaus Zechner
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6

This study aims to model the discourse structure of spontaneous spoken responses within the context of an assessment of English speaking proficiency for non-native speakers. Rhetorical Structure Theory (RST) has been commonly used in the analysis of discourse organization of written texts; however, limited research has been conducted to date on RST annotation and parsing of spoken language, in particular, non-native spontaneous speech. Due to the fact that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we conducted research to obtain RST annotations on non-native spoken responses from a standardized assessment of academic English proficiency. Subsequently, automatic parsers were trained on these annotations to process non-native spontaneous speech. Finally, a set of features were extracted from automatically generated RST trees to evaluate the discourse structure of non-native spontaneous speech, which were then employed to further improve the validity of an automated speech scoring system.
Applying Rhetorical Structure Theory to Student Essays for Providing Automated Writing Feedback
Authors: Shiyan Jiang, Kexin Yang, Ch Suvarna, rakumari, Pooja Casula, Mingtong Zhang, Carolyn Ros{\'e}
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6

We present a package of annotation resources, including annotation guideline, flowchart, and an Intelligent Tutoring System for training human annotators. These resources can be used to apply Rhetorical Structure Theory (RST) to essays written by students in K-12 schools. Furthermore, we highlight the great potential of using RST to provide automated feedback for improving writing quality across genres.
Evaluation of named entity coreference
Authors: Oshin Agarwal, Sanjay Subramanian, Ani Nenkova, Dan Roth
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: coreference-resolution, Coreference Resolution

In many NLP applications like search and information extraction for named entities, it is necessary to find all the mentions of a named entity, some of which appear as pronouns (she, his, etc.) or nominals (the professor, the German chancellor, etc.). It is therefore important that coreference resolution systems are able to link these different types of mentions to the correct entity name. We evaluate state-of-the-art coreference resolution systems for the task of resolving all mentions to named entities. Our analysis reveals that standard coreference metrics do not reflect adequately the requirements in this task: they do not penalize systems for not identifying any mentions by name to an entity and they reward systems even if systems find correctly mentions to the same entity but fail to link these to a proper name (she{--}the student{--}no name). We introduce new metrics for evaluating named entity coreference that address these discrepancies and show that for the comparisons of competitive systems, standard coreference evaluations could give misleading results for this task. We are, however, able to confirm that the state-of-the art system according to traditional evaluations also performs vastly better than other systems on the named entity coreference task.
Neural Coreference Resolution with Limited Lexical Context and Explicit Mention Detection for Oral French
Authors: Lo{\"\i}c Grobol
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: coreference-resolution, Coreference Resolution

We propose an end-to-end coreference resolution system obtained by adapting neural models that have recently improved the state-of-the-art on the OntoNotes benchmark to make them applicable to other paradigms for this task. We report the performances of our system on ANCOR, a corpus of transcribed oral French, for which it constitutes a new baseline with proper evaluation.
Entity Decisions in Neural Language Modelling: Approaches and Problems
Authors: Jenny Kunz, Christian Hardmeier
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Language Modelling

We explore different approaches to explicit entity modelling in language models (LM). We independently replicate two existing models in a controlled setup, introduce a simplified variant of one of the models and analyze their performance in direct comparison. Our results suggest that today{'}s models are limited as several stochastic variables make learning difficult. We show that the most challenging point in the systems is the decision if the next token is an entity token. The low precision and recall for this variable will lead to severe cascading errors. Our own simplified approach dispenses with the need for latent variables and improves the performance in the entity yes/no decision. A standard well-tuned baseline RNN-LM with a larger number of hidden units outperforms all entity-enabled LMs in terms of perplexity.
Cross-lingual NIL Entity Clustering for Low-resource Languages
Authors: Kevin Blissett, Heng Ji
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Clustering

Clustering unlinkable entity mentions across documents in multiple languages (cross-lingual NIL Clustering) is an important task as part of Entity Discovery and Linking (EDL). This task has been largely neglected by the EDL community because it is challenging to outperform simple edit distance or other heuristics based baselines. We propose a novel approach based on encoding the orthographic similarity of the mentions using a Recurrent Neural Network (RNN) architecture. Our model adapts a training procedure from the one-shot facial recognition literature in order to achieve this. We also perform several exploratory probing tasks on our name encodings in order to determine what specific types of information are likely to be encoded by our model. Experiments show our approach provides up to a 6.6{\%} absolute CEAFm F-Score improvement over state-of-the-art methods and successfully captures phonological relations across languages.
Cross-lingual Incongruences in the Annotation of Coreference
Authors: Ekaterina Lapshinova-Koltunski, Sharid Lo{\'a}iciga, Christian Hardmeier, Pauline Krielke
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: coreference-resolution, Coreference Resolution, Translation

In the present paper, we deal with incongruences in English-German multilingual coreference annotation and present automated methods to discover them. More specifically, we automatically detect full coreference chains in parallel texts and analyse discrepancies in their annotations. In doing so, we wish to find out whether the discrepancies rather derive from language typological constraints, from the translation or the actual annotation process. The results of our study contribute to the referential analysis of similarities and differences across languages and support evaluation of cross-lingual coreference annotation. They are also useful for cross-lingual coreference resolution systems and contrastive linguistic studies.
Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque
Authors: Gorka Urbizu, Ander Soraluze, Olatz Arregi
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: coreference-resolution, Coreference Resolution

In this paper, we present a cross-lingual neural coreference resolution system for a less-resourced language such as Basque. To begin with, we build the first neural coreference resolution system for Basque, training it with the relatively small EPEC-KORREF corpus (45,000 words). Next, a cross-lingual coreference resolution system is designed. With this approach, the system learns from a bigger English corpus, using cross-lingual embeddings, to perform the coreference resolution for Basque. The cross-lingual system obtains slightly better results (40.93 F1 CoNLL) than the monolingual system (39.12 F1 CoNLL), without using any Basque language corpus to train it.
The Active-Filler Strategy in a Move-Eager Left-Corner Minimalist Grammar Parser
Authors: Tim Hunter, Milo{\v{s}} Stanojevi{\'c}, Edward Stabler
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Human Parsing, Position

Recent psycholinguistic evidence suggests that human parsing of moved elements is {`}active{'}, and perhaps even {`}hyper-active{'}: it seems that a leftward-moved object is related to a verbal position rapidly, perhaps even before the transitivity information associated with the verb is available to the listener. This paper presents a formal, sound and complete parser for Minimalist Grammars whose search space contains branching points that we can identify as the locus of the decision to perform this kind of active gap-finding. This brings formal models of parsing into closer contact with recent psycholinguistic theorizing than was previously possible.
Priming vs. Inhibition of Optional Infinitival ``to''
Authors: Robin Melnick, Thomas Wasow
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6

The word {``}to{''} that precedes verbs in English infinitives is optional in at least two environments: in what Wasow et al. (2015) previously called the {``}do-be{''} construction, and in the complement of {``}help{''}, which we explore in the present work. In the {``}do-be{''} construction, Wasow et al. found that a preceding infinitival {``}to{''} increases the use of following optional {``}to{''}, but the use of {``}to{''} in the complement of help is reduced following {``}to help{''}. We examine two hypotheses regarding why the same function word is primed by prior use in one construction and inhibited in another. We then test predictions made by the two hypotheses, finding support for one of them.
Simulating Spanish-English Code-Switching: El Modelo Est\'a Generating Code-Switches
Authors: Chara Tsoukala, Stefan L. Frank, Antal Van den Bosch, Jorge Vald{\'e}s Kroff, Mirjam Broersma
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Sentence

Multilingual speakers are able to switch from one language to the other ({``}code-switch{''}) between or within sentences. Because the underlying cognitive mechanisms are not well understood, in this study we use computational cognitive modeling to shed light on the process of code-switching. We employed the Bilingual Dual-path model, a Recurrent Neural Network of bilingual sentence production (Tsoukala et al., 2017), and simulated sentence production in simultaneous Spanish-English bilinguals. Our first goal was to investigate whether the model would code-switch without being exposed to code-switched training input. The model indeed produced code-switches even without any exposure to such input and the patterns of code-switches are in line with earlier linguistic work (Poplack,1980). The second goal of this study was to investigate an auxiliary phrase asymmetry that exists in Spanish-English code-switched production. Using this cognitive model, we examined a possible cause for this asymmetry. To our knowledge, this is the first computational cognitive model that aims to simulate code-switched sentence production.
Surprisal and Interference Effects of Case Markers in Hindi Word Order
Authors: Sidharth Ranjan, Sumeet Agarwal, Rajakrishnan Rajkumar
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Sentence

Based on the Production-Distribution-Comprehension (PDC) account of language processing, we formulate two distinct hypotheses about case marking, word order choices and processing in Hindi. Our first hypothesis is that Hindi tends to optimize for processing efficiency at both lexical and syntactic levels. We quantify the role of case markers in this process. For the task of predicting the reference sentence occurring in a corpus (amidst meaning-equivalent grammatical variants) using a machine learning model, surprisal estimates from an artificial version of the language (i.e., Hindi without any case markers) result in lower prediction accuracy compared to natural Hindi. Our second hypothesis is that Hindi tends to minimize interference due to case markers while ordering preverbal constituents. We show that Hindi tends to avoid placing next to each other constituents whose heads are marked by identical case inflections. Our findings adhere to PDC assumptions and we discuss their implications for language production, learning and universals.
Modeling Hierarchical Syntactic Structures in Morphological Processing
Authors: Yohei Oseki, Charles Yang, Alec Marantz
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Sentence

Sentences are represented as hierarchical syntactic structures, which have been successfully modeled in sentence processing. In contrast, despite the theoretical agreement on hierarchical syntactic structures within words, words have been argued to be computationally less complex than sentences and implemented by finite-state models as linear strings of morphemes, and even the psychological reality of morphemes has been denied. In this paper, extending the computational models employed in sentence processing to morphological processing, we performed a computational simulation experiment where, given incremental surprisal as a linking hypothesis, five computational models with different representational assumptions were evaluated against human reaction times in visual lexical decision experiments available from the English Lexicon Project (ELP), a {``}shared task{''} in the morphological processing literature. The simulation experiment demonstrated that (i) {``}amorphous{''} models without morpheme units underperformed relative to {``}morphous{''} models, (ii) a computational model with hierarchical syntactic structures, Probabilistic Context-Free Grammar (PCFG), most accurately explained human reaction times, and (iii) this performance was achieved on top of surface frequency effects. These results strongly suggest that morphological processing tracks morphemes incrementally from left to right and parses them into hierarchical syntactic structures, contrary to {``}amorphous{''} and finite-state models of morphological processing.
A Modeling Study of the Effects of Surprisal and Entropy in Perceptual Decision Making of an Adaptive Agent
Authors: Pyeong Whan Cho, Richard Lewis
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Decision Making

Processing difficulty in online language comprehension has been explained in terms of surprisal and entropy reduction. Although both hypotheses have been supported by experimental data, we do not fully understand their relative contributions on processing difficulty. To develop a better understanding, we propose a mechanistic model of perceptual decision making that interacts with a simulated task environment with temporal dynamics. The proposed model collects noisy bottom-up evidence over multiple timesteps, integrates it with its top-down expectation, and makes perceptual decisions, producing processing time data directly without relying on any linking hypothesis. Temporal dynamics in the task environment was determined by a simple finite-state grammar, which was designed to create the situations where the surprisal and entropy reduction hypotheses predict different patterns. After the model was trained to maximize rewards, the model developed an adaptive policy and both surprisal and entropy effects were observed especially in a measure reflecting earlier processing.
Modeling Long-Distance Cue Integration in Spoken Word Recognition
Authors: Wednesday Bushong, T. Florian Jaeger
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Spoken Language Understanding

Cues to linguistic categories are distributed across the speech signal. Optimal categorization thus requires that listeners maintain gradient representations of incoming input in order to integrate that information with later cues. There is now evidence that listeners can and do integrate cues that occur far apart in time. Computational models of this integration have however been lacking. We take a first step at addressing this gap by mathematically formalizing four models of how listeners may maintain and use cue information during spoken language understanding and test them on two perception experiments. In one experiment, we find support for rational integration of cues at long distances. In a second, more memory and attention-taxing experiment, we find evidence in favor of a switching model that avoids maintaining detailed representations of cues in memory. These results are a first step in understanding what kinds of mechanisms listeners use for cue integration under different memory and attentional constraints.
Toward a Computational Multidimensional Lexical Similarity Measure for Modeling Word Association Tasks in Psycholinguistics
Authors: Bruno Gaume, Lydia Mai Ho-Dac, Ludovic Tanguy, C{\'e}cile Fabre, B{\'e}n{\'e}dicte Pierrejean, Nabil Hathout, J{\'e}r{\^o}me Farinas, Julien Pinquier, Lola Danet, Patrice P{\'e}ran, Xavier De Boissezon, M{\'e}lanie Jucla
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: General Classification, Semantic Similarity, Semantic Textual Similarity

This paper presents the first results of a multidisciplinary project, the {``}Evolex{''} project, gathering researchers in Psycholinguistics, Neuropsychology, Computer Science, Natural Language Processing and Linguistics. The Evolex project aims at proposing a new data-based inductive method for automatically characterising the relation between pairs of french words collected in psycholinguistics experiments on lexical access. This method takes advantage of several complementary computational measures of semantic similarity. We show that some measures are more correlated than others with the frequency of lexical associations, and that they also differ in the way they capture different semantic relations. This allows us to consider building a multidimensional lexical similarity to automate the classification of lexical associations.