pwc-archive/papers-with-abstracts huggingface dataset:
Memory-bounded Neural Incremental Parsing for Psycholinguistic Prediction
Authors: Lifeng Jin, William Schuler
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Prediction, Sentence

Syntactic surprisal has been shown to have an effect on human sentence processing, and can be predicted from prefix probabilities of generative incremental parsers. Recent state-of-the-art incremental generative neural parsers are able to produce accurate parses and surprisal values but have unbounded stack memory, which may be used by the neural parser to maintain explicit in-order representations of all previously parsed words, inconsistent with results of human memory experiments. In contrast, humans seem to have a bounded working memory, demonstrated by inhibited performance on word recall in multi-clause sentences (Bransford and Franks, 1971), and on center-embedded sentences (Miller and Isard,1964). Bounded statistical parsers exist, but are less accurate than neural parsers in predict-ing reading times. This paper describes a neural incremental generative parser that is able to provide accurate surprisal estimates and can be constrained to use a bounded stack. Results show that the accuracy gains of neural parsers can be reliably extended to psycholinguistic modeling without risk of distortion due to un-bounded working memory.
Advances in Using Grammars with Latent Annotations for Discontinuous Parsing
Authors: Kilian Gebhardt
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

We present new experiments that transfer techniques from Probabilistic Context-free Grammars with Latent Annotations (PCFG-LA) to two grammar formalisms for discontinuous parsing: linear context-free rewriting systems and hybrid grammars. In particular, Dirichlet priors during EM training, ensemble models, and a new nonterminal scheme for hybrid grammars are evaluated. We find that our grammars are more accurate than previous approaches based on discontinuous grammar formalisms and early instances of the discriminative models but inferior to recent discriminative parsers.
Lexicalization of Probabilistic Linear Context-free Rewriting Systems
Authors: Richard M{\"o}rbitz, Thomas Ruprecht
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

In the field of constituent parsing, probabilistic grammar formalisms have been studied to model the syntactic structure of natural language. More recently, approaches utilizing neural models gained lots of traction in this field, as they achieved accurate results at high speed. We aim for a symbiosis between probabilistic linear context-free rewriting systems (PLCFRS) as a probabilistic grammar formalism and neural models to get the best of both worlds: the interpretability of grammars, and the speed and accuracy of neural models. To combine these two, we consider the approach of supertagging that requires lexicalized grammar formalisms. Here, we present a procedure which turns any PLCFRS G into an equivalent lexicalized PLCFRS G{'}. The derivation trees in G{'} are then mapped to equivalent derivations in G. Our construction for G{'} preserves the probability assignment and does not increase parsing complexity compared to G.
Span-Based LCFRS-2 Parsing
Authors: Milo{\v{s}} Stanojevi{\'c}, Mark Steedman
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Sentence

The earliest models for discontinuous constituency parsers used mildly context-sensitive grammars, but the fashion has changed in recent years to grammar-less transition-based parsers that use strong neural probabilistic models to greedily predict transitions. We argue that grammar-based approaches still have something to contribute on top of what is offered by transition-based parsers. Concretely, by using a grammar formalism to restrict the space of possible trees we can use dynamic programming parsing algorithms for exact search for the most probable tree. Previous chart-based parsers for discontinuous formalisms used probabilistically weak generative models. We instead use a span-based discriminative neural model that preserves the dynamic programming properties of the chart parsers. Our parser does not use an explicit grammar, but it does use explicit grammar formalism constraints: we generate only trees that are within the LCFRS-2 formalism. These properties allow us to construct a new parsing algorithm that runs in lower worst-case time complexity of O(l n{\^{}}4 +n{\^{}}6), where $n$ is the sentence length and $l$ is the number of unique non-terminal labels. This parser is efficient in practice, provides best results among chart-based parsers, and is competitive with the best transition based parsers. We also show that the main bottleneck for further improvement in performance is in the restriction of fan-out to degree 2. We show that well-nestedness is helpful in speeding up parsing, but lowers accuracy.
Statistical Deep Parsing for Spanish Using Neural Networks
Authors: Luis Chiruzzo, Dina Wonsever
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

This paper presents the development of a deep parser for Spanish that uses a HPSG grammar and returns trees that contain both syntactic and semantic information. The parsing process uses a top-down approach implemented using LSTM neural networks, and achieves good performance results in terms of syntactic constituency and dependency metrics, and also SRL. We describe the grammar, corpus and implementation of the parser. Our process outperforms a CKY baseline and other Spanish parsers in terms of global metrics and also for some specific Spanish phenomena, such as clitics reduplication and relative referents.
Overview of the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies
Authors: Gosse Bouma, Djam{\'e} Seddah, Daniel Zeman
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

This overview introduces the task of parsing into enhanced universal dependencies, describes the datasets used for training and evaluation, and evaluation metrics. We outline various approaches and discuss the results of the shared task.
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
Authors: Jenna Kanerva, Filip Ginter, Sampo Pyysalo
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Lemmatization

We present the approach of the TurkuNLP group to the IWPT 2020 shared task on Multilingual Parsing into Enhanced Universal Dependencies. The task involves 28 treebanks in 17 different languages and requires parsers to generate graph structures extending on the basic dependency trees. Our approach combines language-specific BERT models, the UDify parser, neural sequence-to-sequence lemmatization and a graph transformation approach encoding the enhanced structure into a dependency tree. Our submission averaged 84.5{\%} ELAS, ranking first in the shared task. We make all methods and resources developed for this study freely available under open licenses from https://turkunlp.org.
Hybrid Enhanced Universal Dependencies Parsing
Authors: Johannes Heinecke
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Dependency Parsing

This paper describes our system to predict enhanced dependencies for Universal Dependencies (UD) treebanks, which ranked 2nd in the Shared Task on Enhanced Dependency Parsing with an average ELAS of 82.60{\%}. Our system uses a hybrid two-step approach. First, we use a graph-based parser to extract a basic syntactic dependency tree. Then, we use a set of linguistic rules which generate the enhanced dependencies for the syntactic tree. The application of these rules is optimized using a classifier which predicts their suitability in the given context. A key advantage of this approach is its language independence, as rules rely solely on dependency trees and UPOS tags which are shared across all languages.
Adaptation of Multilingual Transformer Encoder for Robust Enhanced Universal Dependency Parsing
Authors: Han He, Jinho D. Choi
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Dependency Parsing

This paper presents our enhanced dependency parsing approach using transformer encoders, coupled with a simple yet powerful ensemble algorithm that takes advantage of both tree and graph dependency parsing. Two types of transformer encoders are compared, a multilingual encoder and language-specific encoders. Our dependency tree parsing (DTP) approach generates only primary dependencies to form trees whereas our dependency graph parsing (DGP) approach handles both primary and secondary dependencies to form graphs. Since DGP does not guarantee the generated graphs are acyclic, the ensemble algorithm is designed to add secondary arcs predicted by DGP to primary arcs predicted by DTP. Our results show that models using the multilingual encoder outperform ones using the language specific encoders for most languages. The ensemble models generally show higher labeled attachment score on enhanced dependencies (ELAS) than the DTP and DGP models. As the result, our best models rank the third place on the macro-average ELAS over 17 languages.
Linear Neural Parsing and Hybrid Enhancement for Enhanced Universal Dependencies
Authors: Giuseppe Attardi, Daniele Sartiano, Maria Simi
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Dependency Parsing

To accomplish the shared task on dependency parsing we explore the use of a linear transition-based neural dependency parser as well as a combination of three of them by means of a linear tree combination algorithm. We train separate models for each language on the shared task data. We compare our base parser with two biaffine parsers and also present an ensemble combination of all five parsers, which achieves an average UAS 1.88 point lower than the top official submission. For producing the enhanced dependencies, we exploit a hybrid approach, coupling an algorithmic graph transformation of the dependency tree with predictions made by a multitask machine learning model.
How Much of Enhanced UD Is Contained in UD?
Authors: Adam Ek, Jean-Philippe Bernardy
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

In this paper, we present the submission of team CLASP to the IWPT 2020 Shared Task on parsing enhanced universal dependencies. We develop a tree-to-graph transformation algorithm based on dependency patterns. This algorithm can transform gold UD trees to EUD graphs with an ELAS score of 81.55 and a EULAS score of 96.70. These results show that much of the information needed to construct EUD graphs from UD trees are present in the UD trees. Coupled with a standard UD parser, the method applies to the official test data and yields and ELAS score of 67.85 and a EULAS score is 80.18.
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Authors: Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev, Elham Pejhan, Joakim Nivre
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Sentence

We present K{\o}psala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for everything but enhanced graph parsing, and for the latter, a transition-based graph parser adapted from Che et al. (2019). We train a single enhanced parser model per language, using gold sentence splitting and tokenization for training, and rely only on tokenized surface forms and multilingual BERT for encoding. While a bug introduced just before submission resulted in a severe drop in precision, its post-submission fix would bring us to 4th place in the official ranking, according to average ELAS. Our parser demonstrates that a unified pipeline is effective for both Meaning Representation Parsing and Enhanced Universal Dependencies.
RobertNLP at the IWPT 2020 Shared Task: Surprisingly Simple Enhanced UD Parsing for English
Authors: Stefan Gr{\"u}newald, Annemarie Friedrich
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Sentence

This paper presents our system at the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies. Using a biaffine classifier architecture (Dozat and Manning, 2017) which operates directly on finetuned RoBERTa embeddings, our parser generates enhanced UD graphs by predicting the best dependency label (or absence of a dependency) for each pair of tokens in the sentence. We address label sparsity issues by replacing lexical items in relations with placeholders at prediction time, later retrieving them from the parse in a rule-based fashion. In addition, we ensure structural graph constraints using a simple set of heuristics. On the English blind test data, our system achieves a very high parsing accuracy, ranking 1st out of 10 with an ELAS F1 score of 88.94{\%}.
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN
Authors: Ebrahim Ansari, Amittai Axelrod, Nguyen Bach, Ond{\v{r}}ej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir Durrani, Marcello Federico, Christian Federmann, Jiatao Gu, Fei Huang, Kevin Knight, Xutai Ma, Ajay Nagesh, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Xing Shi, Sebastian St{\"u}ker, Marco Turchi, Alex Waibel, er, Changhan Wang
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Translation

The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation. A total of teams participated in at least one of the tracks. This paper introduces each track{'}s goal, data and evaluation metrics, and reports the results of the received submissions.
Start-Before-End and End-to-End: Neural Speech Translation by AppTek and RWTH Aachen University
Authors: Parnia Bahar, Patrick Wilken, Tamer Alkhouli, Andreas Guta, Pavel Golik, Evgeny Matusov, Christian Herold
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Machine Translation, NMT, speech-recognition, Speech Recognition, Translation

AppTek and RWTH Aachen University team together to participate in the offline and simultaneous speech translation tracks of IWSLT 2020. For the offline task, we create both cascaded and end-to-end speech translation systems, paying attention to careful data selection and weighting. In the cascaded approach, we combine high-quality hybrid automatic speech recognition (ASR) with the Transformer-based neural machine translation (NMT). Our end-to-end direct speech translation systems benefit from pretraining of adapted encoder and decoder components, as well as synthetic data and fine-tuning and thus are able to compete with cascaded systems in terms of MT quality. For simultaneous translation, we utilize a novel architecture that makes dynamic decisions, learned from parallel data, to determine when to continue feeding on input or generate output words. Experiments with speech and text input show that even at low latency this architecture leads to superior translation results.
KIT's IWSLT 2020 SLT Translation System
Authors: Ngoc-Quan Pham, Felix Schneider, Tuan-Nam Nguyen, Thanh-Le Ha, Thai Son Nguyen, Maximilian Awiszus, Sebastian St{\"u}ker, Alex Waibel, er
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Translation

This paper describes KIT{'}s submissions to the IWSLT2020 Speech Translation evaluation campaign. We first participate in the simultaneous translation task, in which our simultaneous models are Transformer based and can be efficiently trained to obtain low latency with minimized compromise in quality. On the offline speech translation task, we applied our new Speech Transformer architecture to end-to-end speech translation. The obtained model can provide translation quality which is competitive to a complicated cascade. The latter still has the upper hand, thanks to the ability to transparently access to the transcription, and resegment the inputs to avoid fragmentation.
End-to-End Simultaneous Translation System for IWSLT2020 Using Modality Agnostic Meta-Learning
Authors: Hou Jeung Han, Mohd Abbas Zaidi, Sathish Reddy Indurthi, Nikhil Kumar Lakumarapu, Beomseok Lee, Sangha Kim
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Meta-Learning, Speech-to-Text, Translation

In this paper, we describe end-to-end simultaneous speech-to-text and text-to-text translation systems submitted to IWSLT2020 online translation challenge. The systems are built by adding wait-k and meta-learning approaches to the Transformer architecture. The systems are evaluated on different latency regimes. The simultaneous text-to-text translation achieved a BLEU score of 26.38 compared to the competition baseline score of 14.17 on the low latency regime (Average latency {\mbox{$\leq$}} 3). The simultaneous speech-to-text system improves the BLEU score by 7.7 points over the competition baseline for the low latency regime (Average Latency {\mbox{$\leq$}} 1000).
DiDi Labs' End-to-end System for the IWSLT 2020 Offline Speech TranslationTask
Authors: Arkady Arkhangorodsky, Yiqi Huang, Amittai Axelrod
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Translation

This paper describes the system that was submitted by DiDi Labs to the offline speech translation task for IWSLT 2020. We trained an end-to-end system that translates audio from English TED talks to German text, without producing intermediate English text. We use the S-Transformer architecture and train using the MuSTC dataset. We also describe several additional experiments that were attempted, but did not yield improved results.
End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning
Authors: Nikhil Kumar Lakumarapu, Beomseok Lee, Sathish Reddy Indurthi, Hou Jeung Han, Mohd Abbas Zaidi, Sangha Kim
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Data Augmentation, Machine Translation, Meta-Learning, speech-recognition, Speech Recognition, Speech-to-Text, Speech-to-Text Translation, Translation

In this paper, we describe the system submitted to the IWSLT 2020 Offline Speech Translation Task. We adopt the Transformer architecture coupled with the meta-learning approach to build our end-to-end Speech-to-Text Translation (ST) system. Our meta-learning approach tackles the data scarcity of the ST task by leveraging the data available from Automatic Speech Recognition (ASR) and Machine Translation (MT) tasks. The meta-learning approach combined with synthetic data augmentation techniques improves the model performance significantly and achieves BLEU scores of 24.58, 27.51, and 27.61 on IWSLT test 2015, MuST-C test, and Europarl-ST test sets respectively.
SRPOL's System for the IWSLT 2020 End-to-End Speech Translation Task
Authors: Tomasz Potapczyk, Pawel Przybysz
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Translation

We took part in the offline End-to-End English to German TED lectures translation task. We based our solution on our last year{'}s submission. We used a slightly altered Transformer architecture with ResNet-like convolutional layer preparing the audio input to Transformer encoder. To improve the model{'}s quality of translation we introduced two regularization techniques and trained on machine translated Librispeech corpus in addition to iwslt-corpus, TEDLIUM2 andMust{\_}C corpora. Our best model scored almost 3 BLEU higher than last year{'}s model. To segment 2020 test set we used exactly the same procedure as last year.
The University of Helsinki Submission to the IWSLT2020 Offline SpeechTranslation Task
Authors: Ra{\'u}l V{\'a}zquez, Mikko Aulamo, Umut Sulubacak, J{\"o}rg Tiedemann
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Transfer Learning, Translation

This paper describes the University of Helsinki Language Technology group{'}s participation in the IWSLT 2020 offline speech translation task, addressing the translation of English audio into German text. In line with this year{'}s task objective, we train both cascade and end-to-end systems for spoken language translation. We opt for an end-to-end multitasking architecture with shared internal representations and a cascade approach that follows a standard procedure consisting of ASR, correction, and MT stages. We also describe the experiments that served as a basis for the submitted systems. Our experiments reveal that multitasking training with shared internal representations is not only possible but allows for knowledge-transfer across modalities.
The AFRL IWSLT 2020 Systems: Work-From-Home Edition
Authors: Brian Ore, Eric Hansen, Tim Anderson, Jeremy Gwinnup
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Action Detection, Activity Detection, Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, Machine Translation, Segmentation, Sentence, Sentence segmentation, speech-recognition, Speech Recognition, Translation

This report summarizes the Air Force Research Laboratory (AFRL) submission to the offline spoken language translation (SLT) task as part of the IWSLT 2020 evaluation campaign. As in previous years, we chose to adopt the cascade approach of using separate systems to perform speech activity detection, automatic speech recognition, sentence segmentation, and machine translation. All systems were neural based, including a fully-connected neural network for speech activity detection, a Kaldi factorized time delay neural network with recurrent neural network (RNN) language model rescoring for speech recognition, a bidirectional RNN with attention mechanism for sentence segmentation, and transformer networks trained with OpenNMT and Marian for machine translation. Our primary submission yielded BLEU scores of 21.28 on tst2019 and 23.33 on tst2020.
LIT Team's System Description for Japanese-Chinese Machine Translation Task in IWSLT 2020
Authors: Yimeng Zhuang, Yuan Zhang, Lijie Wang
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Machine Translation, Translation, Word Embeddings

This paper describes the LIT Team{'}s submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction. Our system is based on the organizers{'} baseline system, but we do more works on improving the Transform baseline system by elaborate data pre-processing. We manage to obtain significant improvements, and this paper aims to share some data processing experiences in this translation task. Large-scale back-translation on monolingual corpus is also investigated. In addition, we also try shared and exclusive word embeddings, compare different granularity of tokens like sub-word level. Our Japanese-to-Chinese translation system achieves a performance of BLEU=34.0 and ranks 2nd among all participating systems.
OPPO's Machine Translation System for the IWSLT 2020 Open Domain Translation Task
Authors: Qian Zhang, Xiaopu Li, Dawei Dang, Tingxun Shi, Di Ai, Zhengshan Xue, Jie Hao
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Machine Translation, Translation

In this paper, we demonstrate our machine translation system applied for the Chinese-Japanese bidirectional translation task (aka. open domain translation task) for the IWSLT 2020. Our model is based on Transformer (Vaswani et al., 2017), with the help of many popular, widely proved effective data preprocessing and augmentation methods. Experiments show that these methods can improve the baseline model steadily and significantly.
Character Mapping and Ad-hoc Adaptation: Edinburgh's IWSLT 2020 Open Domain Translation System
Authors: Pin-zhen Chen, Nikolay Bogoychev, Ulrich Germann
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Machine Translation, Translation

This paper describes the University of Edinburgh{'}s neural machine translation systems submitted to the IWSLT 2020 open domain Japanese$\leftrightarrow$Chinese translation task. On top of commonplace techniques like tokenisation and corpus cleaning, we explore character mapping and unsupervised decoding-time adaptation. Our techniques focus on leveraging the provided data, and we show the positive impact of each technique through the gradual improvement of BLEU.