pwc-archive/papers-with-abstracts huggingface dataset:
Alibaba Submission for WMT18 Quality Estimation Task
Authors: Jiayi Wang, Kai Fan, Bo Li, Fengming Zhou, Boxing Chen, Yangbin Shi, Luo Si
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Automatic Post-Editing, Language Modeling, Language Modelling, Machine Translation, Sentence, Translation

The goal of WMT 2018 Shared Task on Translation Quality Estimation is to investigate automatic methods for estimating the quality of machine translation results without reference translations. This paper presents the QE Brain system, which proposes the neural Bilingual Expert model as a feature extractor based on conditional target language model with a bidirectional transformer and then processes the semantic representations of source and the translation output with a Bi-LSTM predictive model for automatic quality estimation. The system has been applied to the sentence-level scoring and ranking tasks as well as the word-level tasks for finding errors for each word in translations. An extensive set of experimental results have shown that our system outperformed the best results in WMT 2017 Quality Estimation tasks and obtained top results in WMT 2018.
Quality Estimation with Force-Decoded Attention and Cross-lingual Embeddings
Authors: Elizaveta Yankovskaya, Andre T{\"a}ttar, Mark Fishel
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, regression, Sentence, Translation

This paper describes the submissions of the team from the University of Tartu for the sentence-level Quality Estimation shared task of WMT18. The proposed models use features based on attention weights of a neural machine translation system and cross-lingual phrase embeddings as input features of a regression model. Two of the proposed models require only a neural machine translation system with an attention mechanism with no additional resources. Results show that combining neural networks and baseline features leads to significant improvements over the baseline features alone.
A Transformer-Based Multi-Source Automatic Post-Editing System
Authors: Santanu Pal, Nico Herbig, Antonio Kr{\"u}ger, Josef van Genabith
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Automatic Post-Editing, Machine Translation, NMT, Sentence, Translation

This paper presents our English{--}German Automatic Post-Editing (APE) system submitted to the APE Task organized at WMT 2018 (Chatterjee et al., 2018). The proposed model is an extension of the transformer architecture: two separate self-attention-based encoders encode the machine translation output (mt) and the source (src), followed by a joint encoder that attends over a combination of these two encoded sequences (encsrc and encmt) for generating the post-edited sentence. We compare this multi-source architecture (i.e, {src, mt} → pe) to a monolingual transformer (i.e., mt → pe) model and an ensemble combining the multi-source {src, mt} → pe and single-source mt → pe models. For both the PBSMT and the NMT task, the ensemble yields the best results, followed by the multi-source model and last the single-source approach. Our best model, the ensemble, achieves a BLEU score of 66.16 and 74.22 for the PBSMT and NMT task, respectively.
DFKI-MLT System Description for the WMT18 Automatic Post-editing Task
Authors: Daria Pylypenko, Raphael Rubino
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Automatic Post-Editing, Machine Translation, Translation, Word Embeddings

This paper presents the Automatic Post-editing (APE) systems submitted by the DFKI-MLT group to the WMT{'}18 APE shared task. Three monolingual neural sequence-to-sequence APE systems were trained using target-language data only: one using an attentional recurrent neural network architecture and two using the attention-only (\textit{transformer}) architecture. The training data was composed of machine translated (MT) output used as source to the APE model aligned with their manually post-edited version or reference translation as target. We made use of the provided training sets only and trained APE models applicable to phrase-based and neural MT outputs. Results show better performances reached by the attention-only model over the recurrent one, significant improvement over the baseline when post-editing phrase-based MT output but degradation when applied to neural MT output.
Multi-encoder Transformer Network for Automatic Post-Editing
Authors: Jaehun Shin, Jong-Hyeok Lee
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Automatic Post-Editing, Decoder, Machine Translation, NMT, Translation

This paper describes the POSTECH{'}s submission to the WMT 2018 shared task on Automatic Post-Editing (APE). We propose a new neural end-to-end post-editing model based on the transformer network. We modified the encoder-decoder attention to reflect the relation between the machine translation output, the source and the post-edited translation in APE problem. Experiments on WMT17 English-German APE data set show an improvement in both TER and BLEU score over the best result of WMT17 APE shared task. Our primary submission achieves -4.52 TER and +6.81 BLEU score on PBSMT task and -0.13 TER and +0.40 BLEU score for NMT task compare to the baseline.
Multi-source transformer with combined losses for automatic post editing
Authors: Amirhossein Tebbifakhr, Ruchit Agrawal, Matteo Negri, Marco Turchi
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Automatic Post-Editing, Machine Translation, NMT, Sentence, Translation

Recent approaches to the Automatic Post-editing (APE) of Machine Translation (MT) have shown that best results are obtained by neural multi-source models that correct the raw MT output by also considering information from the corresponding source sentence. To this aim, we present for the first time a neural multi-source APE model based on the Transformer architecture. Moreover, we employ sequence-level loss functions in order to avoid exposure bias during training and to be consistent with the automatic evaluation metrics used for the task. These are the main features of our submissions to the WMT 2018 APE shared task, where we participated both in the PBSMT subtask (i.e. the correction of MT outputs from a phrase-based system) and in the NMT subtask (i.e. the correction of neural outputs). In the first subtask, our system improves over the baseline up to -5.3 TER and +8.23 BLEU points ranking second out of 11 submitted runs. In the second one, characterized by the higher quality of the initial translations, we report lower but statistically significant gains (up to -0.38 TER and +0.8 BLEU), ranking first out of 10 submissions.
The Speechmatics Parallel Corpus Filtering System for WMT18
Authors: Tom Ash, Remi Francis, Will Williams
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation

Our entry to the parallel corpus filtering task uses a two-step strategy. The first step uses a series of pragmatic hard {`}rules{'} to remove the worst example sentences. This first step reduces the effective corpus size down from the initial 1 billion to 160 million tokens. The second step uses four different heuristics weighted to produce a score that is then used for further filtering down to 100 or 10 million tokens. Our final system produces competitive results without requiring excessive fine tuning to the exact task or language pair. The first step in isolation provides a very fast filter that gives most of the gains of the final system.
STACC, OOV Density and N-gram Saturation: Vicomtech's Participation in the WMT 2018 Shared Task on Parallel Corpus Filtering
Authors: Andoni Azpeitia, Thierry Etchegoyhen, Eva Mart{\'\i}nez Garcia
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Outlier Detection, Sentence

We describe Vicomtech{'}s participation in the WMT 2018 Shared Task on parallel corpus filtering. We aimed to evaluate a simple approach to the task, which can efficiently process large volumes of data and can be easily deployed for new datasets in different language pairs and domains. We based our approach on STACC, an efficient and portable method for parallel sentence identification in comparable corpora. To address the specifics of the corpus filtering task, which features significant volumes of noisy data, the core method was expanded with a penalty based on the amount of unknown words in sentence pairs. Additionally, we experimented with a complementary data saturation method based on source sentence n-grams, with the goal of demoting parallel sentence pairs that do not contribute significant amounts of yet unobserved n-grams. Our approach requires no prior training and is highly efficient on the type of large datasets featured in the corpus filtering task. We achieved competitive results with this simple and portable method, ranking in the top half among competing systems overall.
A hybrid pipeline of rules and machine learning to filter web-crawled parallel corpora
Authors: Eduard Barbu, Verginica Barbu Mititelu
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: BIG-bench Machine Learning, Machine Translation, regression, Translation, Word Alignment

A hybrid pipeline comprising rules and machine learning is used to filter a noisy web English-German parallel corpus for the Parallel Corpus Filtering task. The core of the pipeline is a module based on the logistic regression algorithm that returns the probability that a translation unit is accepted. The training set for the logistic regression is created by automatic annotation. The quality of the automatic annotation is estimated by manually labeling the training set.
Coverage and Cynicism: The AFRL Submission to the WMT 2018 Parallel Corpus Filtering Task
Authors: Grant Erdmann, Jeremy Gwinnup
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation

The WMT 2018 Parallel Corpus Filtering Task aims to test various methods of filtering a noisy parallel corpus, to make it useful for training machine translation systems. We describe the AFRL submissions, including their preprocessing methods and quality metrics. Numerical results indicate relative benefits of different options and show where our methods are competitive.
MAJE Submission to the WMT2018 Shared Task on Parallel Corpus Filtering
Authors: Marina Fomicheva, Jes{\'u}s Gonz{\'a}lez-Rubio
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation

This paper describes the participation of Webinterpret in the shared task on parallel corpus filtering at the Third Conference on Machine Translation (WMT 2018). The paper describes the main characteristics of our approach and discusses the results obtained on the data sets published for the shared task.
An Unsupervised System for Parallel Corpus Filtering
Authors: Viktor Hangya, Alex Fraser, er
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Machine Translation, Sentence, Sentence Embedding, Text Simplification, Translation, Word Embeddings

In this paper we describe LMU Munich{'}s submission for the \textit{WMT 2018 Parallel Corpus Filtering} shared task which addresses the problem of cleaning noisy parallel corpora. The task of mining and cleaning parallel sentences is important for improving the quality of machine translation systems, especially for low-resource languages. We tackle this problem in a fully unsupervised fashion relying on bilingual word embeddings created without any bilingual signal. After pre-filtering noisy data we rank sentence pairs by calculating bilingual sentence-level similarities and then remove redundant data by employing monolingual similarity as well. Our unsupervised system achieved good performance during the official evaluation of the shared task, scoring only a few BLEU points behind the best systems, while not requiring any parallel training data.
The JHU Parallel Corpus Filtering Systems for WMT 2018
Authors: Huda Khayrallah, Hainan Xu, Philipp Koehn
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Language Modeling, Language Modelling, Machine Translation, Outlier Detection, Sentence

This work describes our submission to the WMT18 Parallel Corpus Filtering shared task. We use a slightly modified version of the Zipporah Corpus Filtering toolkit (Xu and Koehn, 2017), which computes an adequacy score and a fluency score on a sentence pair, and use a weighted sum of the scores as the selection criteria. This work differs from Zipporah in that we experiment with using the noisy corpus to be filtered to compute the combination weights, and thus avoids generating synthetic data as in standard Zipporah.
Measuring sentence parallelism using Mahalanobis distances: The NRC unsupervised submissions to the WMT18 Parallel Corpus Filtering shared task
Authors: Patrick Littell, Samuel Larkin, Darlene Stewart, Michel Simard, Cyril Goutte, Chi-kiu Lo
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Anomaly Detection, Machine Translation, Sentence

The WMT18 shared task on parallel corpus filtering (Koehn et al., 2018b) challenged teams to score sentence pairs from a large high-recall, low-precision web-scraped parallel corpus (Koehn et al., 2018a). Participants could use existing sample corpora (e.g. past WMT data) as a supervisory signal to learn what a {``}clean{''} corpus looks like. However, in lower-resource situations it often happens that the target corpus of the language is the \textit{only} sample of parallel text in that language. We therefore made several unsupervised entries, setting ourselves an additional constraint that we not utilize the additional clean parallel corpora. One such entry fairly consistently scored in the top ten systems in the 100M-word conditions, and for one task{---}translating the European Medicines Agency corpus (Tiedemann, 2009){---}scored among the best systems even in the 10M-word conditions.
Accurate semantic textual similarity for cleaning noisy parallel corpora using semantic machine translation evaluation metric: The NRC supervised submissions to the Parallel Corpus Filtering task
Authors: Chi-kiu Lo, Michel Simard, Darlene Stewart, Samuel Larkin, Cyril Goutte, Patrick Littell
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Semantic Textual Similarity, Translation

We present our semantic textual similarity approach in filtering a noisy web crawled parallel corpus using YiSi{---}a novel semantic machine translation evaluation metric. The systems mainly based on this supervised approach perform well in the WMT18 Parallel Corpus Filtering shared task (4th place in 100-million-word evaluation, 8th place in 10-million-word evaluation, and 6th place overall, out of 48 submissions). In fact, our best performing system{---}NRC-yisi-bicov is one of the only four submissions ranked top 10 in both evaluations. Our submitted systems also include some initial filtering steps for scaling down the size of the test corpus and a final redundancy removal step for better semantic and token coverage of the filtered corpus. In this paper, we also describe our unsuccessful attempt in automatically synthesizing a noisy parallel development corpus for tuning the weights to combine different parallelism and fluency features.
Alibaba Submission to the WMT18 Parallel Corpus Filtering Task
Authors: Jun Lu, Xiaoyu Lv, Yangbin Shi, Boxing Chen
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Diversity, Machine Translation, Sentence, Translation, Word Alignment

This paper describes the Alibaba Machine Translation Group submissions to the WMT 2018 Shared Task on Parallel Corpus Filtering. While evaluating the quality of the parallel corpus, the three characteristics of the corpus are investigated, i.e. 1) the bilingual/translation quality, 2) the monolingual quality and 3) the corpus diversity. Both rule-based and model-based methods are adapted to score the parallel sentence pairs. The final parallel corpus filtering system is reliable, easy to build and adapt to other language pairs.
UTFPR at WMT 2018: Minimalistic Supervised Corpora Filtering for Machine Translation
Authors: Gustavo Paetzold
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Binary Classification, Classification, General Classification, Language Modeling, Language Modelling, Machine Translation, regression, Translation

We present the UTFPR systems at the WMT 2018 parallel corpus filtering task. Our supervised approach discerns between good and bad translations by training classic binary classification models over an artificially produced binary classification dataset derived from a high-quality translation set, and a minimalistic set of 6 semantic distance features that rely only on easy-to-gather resources. We rank translations by their probability for the {``}good{''} label. Our results show that logistic regression pairs best with our approach, yielding more consistent results throughout the different settings evaluated.
The ILSP/ARC submission to the WMT 2018 Parallel Corpus Filtering Shared Task
Authors: Vassilis Papavassiliou, Sokratis Sofianopoulos, Prokopis Prokopidis, Stelios Piperidis
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: ARC, Clustering, Language Modeling, Language Modelling, Machine Translation, Outlier Detection, Sentence

This paper describes the submission of the Institute for Language and Speech Processing/Athena Research and Innovation Center (ILSP/ARC) for the WMT 2018 Parallel Corpus Filtering shared task. We explore several properties of sentences and sentence pairs that our system explored in the context of the task with the purpose of clustering sentence pairs according to their appropriateness in training MT systems. We also discuss alternative methods for ranking the sentence pairs of the most appropriate clusters with the aim of generating the two datasets (of 10 and 100 million words as required in the task) that were evaluated. By summarizing the results of several experiments that were carried out by the organizers during the evaluation phase, our submission achieved an average BLEU score of 26.41, even though it does not make use of any language-specific resources like bilingual lexica, monolingual corpora, or MT output, while the average score of the best participant system was 27.91.
SYSTRAN Participation to the WMT2018 Shared Task on Parallel Corpus Filtering
Authors: MinhQuang Pham, Josep Crego, Jean Senellart
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Feature Engineering, Machine Translation, Sentence, Sentence Similarity, Translation

This paper describes the participation of SYSTRAN to the shared task on parallel corpus filtering at the Third Conference on Machine Translation (WMT 2018). We participate for the first time using a neural sentence similarity classifier which aims at predicting the relatedness of sentence pairs in a multilingual context. The paper describes the main characteristics of our approach and discusses the results obtained on the data sets published for the shared task.
Tilde's Parallel Corpus Filtering Methods for WMT 2018
Authors: M{\=a}rcis Pinnis
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation, Transliteration, Word Alignment

The paper describes parallel corpus filtering methods that allow reducing noise of noisy {``}parallel{''} corpora from a level where the corpora are not usable for neural machine translation training (i.e., the resulting systems fail to achieve reasonable translation quality; well below 10 BLEU points) up to a level where the trained systems show decent (over 20 BLEU points on a 10 million word dataset and up to 30 BLEU points on a 100 million word dataset). The paper also documents Tilde{'}s submissions to the WMT 2018 shared task on parallel corpus filtering.
The RWTH Aachen University Filtering System for the WMT 2018 Parallel Corpus Filtering Task
Authors: Nick Rossenbach, Jan Rosendahl, Yunsu Kim, Miguel Gra{\c{c}}a, Aman Gokrani, Hermann Ney
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Sentence, Translation

This paper describes the submission of RWTH Aachen University for the Deâ†’En parallel corpus filtering task of the \textit{EMNLP 2018 Third Conference on Machine Translation} (WMT 2018). We use several rule-based, heuristic methods to preselect sentence pairs. These sentence pairs are scored with count-based and neural systems as language and translation models. In addition to single sentence-pair scoring, we further implement a simple redundancy removing heuristic. Our best performing corpus filtering system relies on recurrent neural language models and translation models based on the transformer architecture. A model trained on 10M randomly sampled tokens reaches a performance of 9.2{\%} BLEU on newstest2018. Using our filtering and ranking techniques we achieve 34.8{\%} BLEU.
Prompsit's submission to WMT 2018 Parallel Corpus Filtering shared task
Authors: V{\'\i}ctor M. S{\'a}nchez-Cartagena, Marta Ba{\~n}{\'o}n, Sergio Ortiz-Rojas, Gema Ram{\'\i}rez
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Active Learning, Language Modeling, Language Modelling, Machine Translation

This paper describes Prompsit Language Engineering{'}s submissions to the WMT 2018 parallel corpus filtering shared task. Our four submissions were based on an automatic classifier for identifying pairs of sentences that are mutual translations. A set of hand-crafted hard rules for discarding sentences with evident flaws were applied before the classifier. We explored different strategies for achieving a training corpus with diverse vocabulary and fluent sentences: language model scoring, an active-learning-inspired data selection algorithm and n-gram saturation. Our submissions were very competitive in comparison with other participants on the 100 million word training corpus.
Syntactic Manipulation for Generating more Diverse and Interesting Texts
Authors: Jan Milan Deriu, Mark Cieliebak
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

Natural Language Generation plays an important role in the domain of dialogue systems as it determines how users perceive the system. Recently, deep-learning based systems have been proposed to tackle this task, as they generalize better and require less amounts of manual effort to implement them for new domains. However, deep learning systems usually adapt a very homogeneous sounding writing style which expresses little variation. In this work, we present our system for Natural Language Generation where we control various aspects of the surface realization in order to increase the lexical variability of the utterances, such that they sound more diverse and interesting. For this, we use a Semantically Controlled Long Short-term Memory Network (SC-LSTM), and apply its specialized cell to control various syntactic features of the generated texts. We present an in-depth human evaluation where we show the effects of these surface manipulation on the perception of potential users.
Automated learning of templates for data-to-text generation: comparing rule-based, statistical and neural methods
Authors: Chris van der Lee, Emiel Krahmer, S Wubben, er
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Data-to-Text Generation, Machine Translation, Text Generation, Translation

The current study investigated novel techniques and methods for trainable approaches to data-to-text generation. Neural Machine Translation was explored for the conversion from data to text as well as the addition of extra templatization steps of the data input and text output in the conversion process. Evaluation using BLEU did not find the Neural Machine Translation technique to perform any better compared to rule-based or Statistical Machine Translation, and the templatization method seemed to perform similarly or sometimes worse compared to direct data-to-text conversion. However, the human evaluation metrics indicated that Neural Machine Translation yielded the highest quality output and that the templatization method was able to increase text quality in multiple situations.