wanyu/IteraTeR_full_sent huggingface dataset:
Original:  Our experiments on the WMT19 metrics tasks for all into-English language directions show that somewhat surprisingly, the addition of diverse paraphrases, even those produced by humans, leads to only small, inconsistent changes in BLEU's correlation with human judgments, suggesting that BLEU's ability to correctly exploit multiple references is limited
Revised:  Our experiments on the WMT19 metrics tasks for all into-English language directions show that somewhat surprisingly, the addition of diverse paraphrases, even those produced by humans, leads to only small, inconsistent changes in BLEU's correlation with human judgments, suggesting that BLEU's capacity to correctly exploit multiple references is limited
Edit intent: clarity
Original:  Our experiments on the WMT19 metrics tasks for all into-English language directions show that somewhat surprisingly, the addition of diverse paraphrases, even those produced by humans, leads to only small, inconsistent changes in BLEU's correlation with human judgments, suggesting that BLEU's ability to correctly exploit multiple references is limited
Revised:  Our experiments on the WMT19 metrics tasks for all into-English language directions show that somewhat surprisingly, the addition of diverse paraphrases, even those produced by humans, leads to only small, inconsistent changes in BLEU's correlation with human judgments, suggesting that BLEU's ability to correctly exploit multiple references . Surprisingly, we also find that adequacy appears to be less important, as shown by the high results of a strong sampling approach, which even beats human paraphrases when used with sentence-level BLEU.
Edit intent: meaning-changed
Original:  One way to do so is erasure: a subset of inputs is considered irrelevant if it can be removed without affecting the model prediction. Despite its conceptual simplicity, erasureis not commonly used in practice. First, the objective is generally intractable , and approximate search or leave-one-out estimates are typically used instead;
Revised:  One way to do so is erasure: a subset of inputs is considered irrelevant if it can be removed without affecting the prediction. Though conceptually simple, erasure's objective is intractable and approximate search or leave-one-out estimates are typically used instead;
Edit intent: clarity
Original:  First, the objective is generally intractable , and approximate search or leave-one-out estimates are typically used instead; both approximations may be inaccurate and remain very expensive with modern deep (e.g., BERT-based) NLP models.
Revised:  First, the objective is generally intractable , and approximate search remains expensive with modern deep (e.g., BERT-based) NLP models.
Edit intent: coherence
Original: Pre-trained language models have achieved state-of-the-art results in various natural language processing tasks. Most of them are based on the Transformer architecture , which distinguishes tokens with the token position index of the input sequence.
Revised: Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture , which distinguishes tokens with the token position index of the input sequence.
Edit intent: clarity
Original:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex ,  and token index embeddings.
Revised:  To verify this, we propose a segment-aware Transformer (Segatron) , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex , and token index embeddings.
Edit intent: meaning-changed
Original:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex ,  and token index embeddings. We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks  .
Revised:  To verify this, we propose a segment-aware BERT , by replacing the token position embedding of Transformer with a combination of paragraphindex, sentenceindex , and token . We first introduce the segment-aware mechanism to Transformer-XL, which is a popular Transformer-based language model with memory extension and relative position encoding. We find that our method can further improve the Transformer-XL base model and large model, achieving 17.1 perplexity on the WikiText-103 dataset. We further investigate the pre-training masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks .
Edit intent: meaning-changed
Original:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks  .
Revised:  We pre-trained the SegaBERT on the masked language modeling task with Segatron . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks .
Edit intent: meaning-changed
Original:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks  .
Revised:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that BERT pre-trained model can outperform the original BERT model on various NLP tasks .
Edit intent: clarity
Original:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks  .
Revised:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla Transformer on various NLP tasks .
Edit intent: meaning-changed
Original:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks  .
Revised:  We pre-trained the SegaBERT on the masked language modeling task in BERT but without any affiliated tasks . Experimental results show that our pre-trained model can outperform the original BERT model on various NLP tasks , and outperforms RoBERTa on zero-shot sentence representation learning .
Edit intent: meaning-changed
Original: Pre-trained multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on some source language (typically English) and evaluated on a different target language.
Revised: Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on some source language (typically English) and evaluated on a different target language.
Edit intent: clarity
Original: Pre-trained multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on some source language (typically English) and evaluated on a different target language.
Revised: Pre-trained multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on one source language and evaluated on a different target language.
Edit intent: clarity
Original:  English dev accuracy is often uncorrelated (or even anti-correlated) with target language accuracy, and zero-shot cross-lingual performance varies greatly within the same fine-tuning run and between different fine-tuning runs.
Revised:  English dev accuracy is often uncorrelated (or even anti-correlated) with target language accuracy, and zero-shot performance varies greatly within the same fine-tuning run and between different fine-tuning runs.
Edit intent: clarity
Original:  English dev accuracy is often uncorrelated (or even anti-correlated) with target language accuracy, and zero-shot cross-lingual performance varies greatly within the same fine-tuning run and between different fine-tuning runs.
Revised:  English dev accuracy is often uncorrelated (or even anti-correlated) with target language accuracy, and zero-shot cross-lingual performance varies greatly at different points in the same fine-tuning run and between different fine-tuning runs.
Edit intent: clarity
Original:   We recommend providing oracle scores alongside the zero-shot results: still fine-tune using English  , but choose a checkpoint with the target dev set.
Revised:  These reproducibility issues are also present for other tasks with different pre-trained embeddings (e.g., MLQA with XLM-R). We recommend providing oracle scores alongside the zero-shot results: still fine-tune using English , but choose a checkpoint with the target dev set.
Edit intent: meaning-changed
Original: One key principle for assessing semantic similarity between texts is to measure the degree of semantic overlap of them by considering word-by-word alignment. However, alignment-based approaches are inferior to the generic sentence vectorsin terms of performance.
Revised: One key principle for assessing semantic similarity between texts is to measure the degree of semantic overlap between two texts by considering the word alignment. Such alignment-based approaches are inferior to the generic sentence vectorsin terms of performance.
Edit intent: clarity
Original:  However, alignment-based approaches are inferior to the generic sentence vectorsin terms of performance. We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they do not distinguish word importance and word meaning.
Revised:  However, alignment-based approaches are inferior to the simple cosine similarity between general-purpose sentence vectors. To remedy this, we focus on the fact that the norm of word vectors is a good proxy for word importance, and the angle of them is a good proxy for word similarity. Alignment-based approaches do not distinguish word importance and word meaning.
Edit intent: meaning-changed
Original:  We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they do not distinguish word importance and word meaning. To solve this , we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction , then compute the alignment-based similarity with the help of earth mover's distance .
Revised:  We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they do not distinguish the norm and direction, whereas sentence-vector approaches automatically use the norm as the word importance. Accordingly , we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction , then compute the alignment-based similarity with the help of earth mover's distance .
Edit intent: meaning-changed
Original:  To solve this , we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction , then compute the alignment-based similarity with the help of earth mover's distance . We call the method word rotator's distance (WRD) because direction vectors are aligned by rotation on the unit hypersphere.
Revised:  To solve this , we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction , then compute the alignment-based similarity with the help of earth mover's distance (optimal transport cost), which we refer to as word rotator's distance (WRD) because direction vectors are aligned by rotation on the unit hypersphere.
Edit intent: meaning-changed
Original:  We call the method word rotator's distance (WRD) because direction vectors are aligned by rotation on the unit hypersphere. In addition, to incorporate the advance of cutting edge additive sentence encoders, we propose to re-decompose such sentence vectors into word vectors and use them as inputs to WRD. Empirically, the proposed method outperforms current methods considering the word-by-word alignment including word mover's distance with a big difference; moreover, our method outperforms state-of-the-art additive sentence encoders on the most competitive dataset, STS-benchmark .
Revised:  We call the method word rotator's distance . Furthermore, we demonstrate how to grow the norm and direction of word vectors (vector converter); this is a new systematic approach derived from the sentence-vector estimation methods, which can significantly improve the performance of the proposed method. On several STS benchmarks, our simple proposed methods outperformed not only alignment-based approaches but also strong baselines .
Edit intent: clarity
Original:  Users can interactively visualize and inspect the data transformation and flow of intermediate results in a CNN . CNN Explainer tightly integrates a model overview that summarizes a CNN's structure, and on-demand, dynamic visual explanation views that help users understand the underlying components of CNNs.
Revised:  CNN Explainer tightly integrates a model overview that summarizes a CNN's structure, and on-demand, dynamic visual explanation views that help users understand the underlying components of CNNs.
Edit intent: coherence
Original:  Through smooth transitions across levels of abstraction, our tool enables users to inspect the interplay between low-level operations (e.g., mathematical computations) and high-level outcomes (e. g., class predictions). To better understand our tool's benefits, we conducted a qualitative user study , which shows that CNN Explainer can help users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Revised:  Through smooth transitions across levels of abstraction, our tool enables users to inspect the interplay between low-level operations (e.g., mathematical computations) and high-level model structures. A qualitative user study , which shows that CNN Explainer can help users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Edit intent: clarity
Original:  To better understand our tool's benefits, we conducted a qualitative user study , which shows that CNN Explainer can help users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Revised:  To better understand our tool's benefits, we conducted a qualitative user study shows that CNN Explainer can help users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Edit intent: coherence
Original:  To better understand our tool's benefits, we conducted a qualitative user study , which shows that CNN Explainer can help users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Revised:  To better understand our tool's benefits, we conducted a qualitative user study , which shows that CNN Explainer helps users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use.
Edit intent: coherence
Original:  In this work, we propose a template rewriting method for Natural Language Generation (NLG) , where the number of templates scales only linearly with the number of slots .
Revised:  In this work, we propose a template rewriting method for Natural Language Generation (NLG) using a single domain-independent model across a large number of APIs. First, we propose a schema-guided approach which conditions the generation on a schema describing the API in natural language. Our second method investigates the use of a small number of templates scales only linearly with the number of slots .
Edit intent: meaning-changed
Original:  In this work, we propose a template rewriting method for Natural Language Generation (NLG) , where the number of templates scales only linearly with the number of slots . A set of simple templates is used to convert actions into utterances , which are concatenated to give a semantically correct, but possibly incoherent and ungrammatical utterance.
Revised:  In this work, we propose a template rewriting method for Natural Language Generation (NLG) , where the number of templates scales only linearly with the number of slots , to convey the semantics of the API. To generate utterances for an arbitrary slot combination, a few simple templates are first concatenated to give a semantically correct, but possibly incoherent and ungrammatical utterance.
Edit intent: meaning-changed
Original: Segmentation and (segment ) labeling are generally treated separately in lexical semantics, raising issues due to their close inter-dependence and necessitating joint annotation. We therefore investigate the lexical semantic recognition task of multiword expression segmentation and supersense disambiguation, unifying several previously-disparate styles of lexical semantic annotation.
Revised: Segmentation and (segment ) labeling are generally treated separately , despite their interdependence. We hypothesize that a unified lexical semantic recognition task of multiword expression segmentation and supersense disambiguation, unifying several previously-disparate styles of lexical semantic annotation.
Edit intent: clarity
Original:  We therefore investigate the lexical semantic recognition task of multiword expression segmentation and supersense disambiguation, unifying several previously-disparate styles of lexical semantic annotation. We evaluate a neural CRF model along all annotation axes available in version 4.3 of the STREUSLE corpus: lexical unit segmentation (multiword expressions), word-level syntactic tags, and supersense classes for noun, verb, and preposition/possessive units .
Revised:  We therefore investigate the lexical semantic recognition task is an effective way to encapsulate previously disparate styles of annotation, including multiword expression identification / classification and supersense tagging. Using the STREUSLE corpus, we train a neural CRF model along all annotation axes available in version 4.3 of the STREUSLE corpus: lexical unit segmentation (multiword expressions), word-level syntactic tags, and supersense classes for noun, verb, and preposition/possessive units .
Edit intent: meaning-changed
Original:  As the label set generalizes that of previous tasks ( DiMSUM, PARSEME ), we additionally evaluate how well the model generalizes to those test sets, with encouraging results. By establishing baseline models and evaluation metrics , we pave the way for comprehensive and accurate modeling of lexical semantics  .
Revised:  As the label set generalizes that of previous tasks ( DiMSUM, PARSEME ), we additionally evaluate how well the model generalizes to those test sets, finding that it approaches or surpasses existing models despite training only on STREUSLE. Our work also establishes baseline models and evaluation metrics , we pave the way for comprehensive and accurate modeling of lexical semantics .
Edit intent: meaning-changed
Original:  We present a training strategy for adapting pretrained language models that exploits similarities between TLDR generation and the related tasks of extreme summarization and title generation, which outperforms strong extractive and abstractive summarization baselines.
Revised:  We present a training strategy for adapting pretrained language models that exploits similarities between TLDR generation and the related task of title generation, which outperforms strong extractive and abstractive summarization baselines.
Edit intent: clarity
Original:  To facilitate research on this task, we introduce SciTLDR, a dataset of 3.9K TLDRs . Furthermore, we introduce a novel annotation protocol for scalably curating additional gold summaries by rewriting peer review comments.
Revised:  To facilitate study on this task, we introduce SciTLDR, a dataset of 3.9K TLDRs . Furthermore, we introduce a novel annotation protocol for scalably curating additional gold summaries by rewriting peer review comments.
Edit intent: clarity
Original:  To facilitate research on this task, we introduce SciTLDR, a dataset of 3.9K TLDRs . Furthermore, we introduce a novel annotation protocol for scalably curating additional gold summaries by rewriting peer review comments.
Revised:  To facilitate research on this task, we introduce SciTLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers. SciTLDR contains both author-written and expert-derived TLDRs, where the latter are collected using a novel annotation protocol for scalably curating additional gold summaries by rewriting peer review comments.
Edit intent: meaning-changed
Original:  To facilitate research on this task, we introduce SciTLDR, a dataset of 3.9K TLDRs . Furthermore, we introduce a novel annotation protocol for scalably curating additional gold summaries by rewriting peer review comments. We use this protocol to augment our test set, yielding multiple gold TLDRs for evaluation, which is unlike most recent summarization datasets that assume only one valid gold summary. We present a training strategy for adapting pretrained language models that exploits similarities between TLDR generation and the related task of title generation, which outperforms strong extractive and abstractive summarization baselines .
Revised:  To facilitate research on this task, we introduce SciTLDR, a dataset of 3.9K TLDRs . Furthermore, we introduce a novel annotation protocol that produces high-quality summaries while minimizing annotation burden. We propose CATTS, a simple yet effective learning strategy for generating TLDRs that exploits titles as an auxiliary training signal. CATTS improves upon strong baselines under both automated metrics and human evaluations. Data and code are publicly available at URL
Edit intent: meaning-changed
Original:  We design a series of toy learning problems to investigate the conditions under which such data augmentation is helpful. We show that augmenting with training examples on which the weak feature fails ("counterexamples") does succeed in preventing the model from relying on the weak feature, but often does not succeed in encouraging the model to use the stronger feature in general. We also find in many cases that the number of counterexamples needed to reach a given error rate is independent of the amount of training data, and that this type of data augmentation becomes less effective as the target strong feature becomes harder to learn .
Revised:  We design a series of toy learning problems to test the hypothesis that data augmentation leads models to unlearn weaker heuristics, but not to learn stronger features in their place. We find partial support for this hypothesis: Data augmentation often hurts before it helps, and it is less effective when the preferred strong feature is much more difficult to extract than the competing weak feature .
Edit intent: clarity
Original:  We first show that black-box MT systems can be stolen by querying them with monolingual sentences and training models to imitate their outputs.
Revised:  One setting of particular interest is machine translation (MT), where models have high commercial value and errors can be costly. We investigate possible exploitations of black-box MT systems and explore a preliminary defense against such threats. We first show that black-box MT systems can be stolen by querying them with monolingual sentences and training models to imitate their outputs.
Edit intent: meaning-changed