wanyu/IteraTeR_full_sent huggingface dataset:
Original:  Sequential fine-tuning and multi-task learning are two methods for sharing information, but suffer from problems such as catastrophic URLetting and difficulties in balancing multiple tasks. Additionally, multi-task learning requires simultaneous access to data used for each of the tasks, which does not allow for easy extensions to new tasks on the fly. We propose a new architecture as well as a two-stage learning algorithm that allows us to effectively share knowledge from multiple tasks while avoiding these crucial problems.
Revised:  Sequential fine-tuning and multi-task learning are two methods for sharing information, but suffer from problems such as catastrophic URLetting and difficulties in dataset balancing. To address these shortcomings, we propose AdapterFusion, a new two stage learning algorithm that allows us to effectively share knowledge from multiple tasks while avoiding these crucial problems.
Edit intent: clarity
Original:  We propose a new architecture as well as a two-stage learning algorithm that allows us to effectively share knowledge from multiple tasks while avoiding these crucial problems. In the first stage , we learn task specific parameters  that encapsulate the knowledge from each task .
Revised:  We propose a new architecture as well as a two-stage learning algorithm that allows us to effectively share knowledge from multiple tasks . First, in the knowledge extraction stage we learn task specific parameters that encapsulate the knowledge from each task .
Edit intent: meaning-changed
Original:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning  .
Revised:  We empirically evaluate AdapterFusion on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning .
Edit intent: meaning-changed
Original:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning  .
Revised:  We empirically evaluate our transfer learning approach on 16 diverse NLU tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning .
Edit intent: clarity
Original:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning  .
Revised:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and find that it effectively combines various types of knowledge at different layers of the model. We show that our approach outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning .
Edit intent: meaning-changed
Original:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning  .
Revised:  We empirically evaluate our transfer learning approach on 16 diverse NLP tasks, and show that it outperforms traditional strategies such as full fine-tuning of the model as well as multi-task learning . Our code and adapters are available at AdapterHub.ml .
Edit intent: meaning-changed
Original:  We then analyze two scenarios: 1) inducing  biases for one demographic and avoiding biases for another  , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Revised:  We then analyze two scenarios: 1) inducing negative biases for one demographic and avoiding biases for another , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Edit intent: meaning-changed
Original:  We then analyze two scenarios: 1) inducing  biases for one demographic and avoiding biases for another  , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Revised:  We then analyze two scenarios: 1) inducing biases for one demographic and positive biases for another , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Edit intent: meaning-changed
Original:  We then analyze two scenarios: 1) inducing  biases for one demographic and avoiding biases for another  , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Revised:  We then analyze two scenarios: 1) inducing biases for one demographic and avoiding biases for another demographic , and 2) mitigating biases between demographic pairs (e. g., man and woman).
Edit intent: fluency
Original:  The former scenario gives us a tool for detecting the types of biases present in the model , and the latter is useful for mitigating biases in downstream applications (e. g., dialogue generation). Specifically, our approach facilitates more explainable biases by allowing us to 1) use the relative effectiveness of inducing biases for different demographics as a new dimension for bias evaluation, and 2) discover topics that correspond to demographic inequalities in generated text.
Revised:  The former scenario gives us a tool for detecting the types of biases present in the model . Specifically, we show the effectiveness of our approach at facilitating bias analysis by finding topics that correspond to demographic inequalities in generated text and comparing the relative effectiveness of inducing biases for different demographics as a new dimension for bias evaluation, and 2) discover topics that correspond to demographic inequalities in generated text.
Edit intent: clarity
Original:  Specifically, our approach facilitates more explainable biases by allowing us to 1) use the relative effectiveness of inducing biases for different demographics as a new dimension for bias evaluation, and 2) discover topics that correspond to demographic inequalities in generated text. Furthermore, our mitigation experimentsexemplify our technique 's effectiveness at equalizing the amount of biases across demographics while simultaneously generating less negatively biased text overall.
Revised:  Specifically, our approach facilitates more explainable biases by allowing us to 1) use the relative effectiveness of inducing biases for different demographics . The second scenario is useful for mitigating biases in downstream applications such as dialogue generation. In our experiments, the mitigation technique proves to be effective at equalizing the amount of biases across demographics while simultaneously generating less negatively biased text overall.
Edit intent: meaning-changed
Original: Existing question answering datasets mostly contain homogeneous contexts, based on either textual or visual information alone. On the other hand, digitalization has evolved the nature of reading which often includes integrating information across multiple heterogeneous sources. To bridge the gap between two, we compile a Diverse Visuo-Lingustic Question Answering (DVLQA) challenge corpus, where the task is to derive joint inference about the given image-text modality  in a question answering setting.
Revised: Understanding images and text together is an important aspect of cognition and building advanced Artificial Intelligence (AI) systems. As a community, we have achieved good benchmarks over language and vision domains separately, however joint reasoning is still a challenge for state-of-the-art computer vision and natural language processing (NLP) systems. We propose a novel task to derive joint inference about the given image-text modality in a question answering setting.
Edit intent: meaning-changed
Original:  To bridge the gap between two, we compile a Diverse Visuo-Lingustic Question Answering (DVLQA) challenge corpus, where the task is to derive joint inference about the given image-text modality  in a question answering setting.
Revised:  To bridge the gap between two, we compile a Diverse Visuo-Lingustic Question Answering (DVLQA) challenge corpus, where the task is to derive joint inference about a given image-text modality in a question answering setting.
Edit intent: fluency
Original:  To bridge the gap between two, we compile a Diverse Visuo-Lingustic Question Answering (DVLQA) challenge corpus, where the task is to derive joint inference about the given image-text modality  in a question answering setting.
Revised:  To bridge the gap between two, we compile a Diverse Visuo-Lingustic Question Answering (DVLQA) challenge corpus, where the task is to derive joint inference about the given image-text modality and compile the Visuo-Linguistic Question Answering (VLQA) challenge corpus in a question answering setting.
Edit intent: meaning-changed
Original:  We then develop a modular method which demonstrates slightly better baseline performance and offers more transparency for interpretation of intermediate outputs. However, this is still far behind the human performance, therefore we believe DVLQA will be a challenging benchmark for question answering involving reasoning over visuo-linguistic context.
Revised:  We then develop a modular method which demonstrates slightly better baseline performance , but it is still far behind the human performance, therefore we believe DVLQA will be a challenging benchmark for question answering involving reasoning over visuo-linguistic context.
Edit intent: coherence
Original:  Motivated by both demands, we introduce Cross-lingual Choice of Plausible Alternatives (XCOPA), a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages . We benchmark a range of state-of-the-art models on this novel dataset, revealing that  current methods based on multilingual pretraining and zero-shot fine-tuning transfer suffer from the curse of multilinguality and fall short of performance in monolingual settings by a large margin .
Revised:  Motivated by both demands, we introduce Cross-lingual Choice of Plausible Alternatives (XCOPA), a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages , which includes resource-poor languages like Eastern Apur\'imac Quechua and Haitian Creole. We evaluate a range of state-of-the-art models on this novel dataset, revealing that current methods based on multilingual pretraining and zero-shot fine-tuning transfer suffer from the curse of multilinguality and fall short of performance in monolingual settings by a large margin .
Edit intent: meaning-changed
Original:  We benchmark a range of state-of-the-art models on this novel dataset, revealing that  current methods based on multilingual pretraining and zero-shot fine-tuning transfer suffer from the curse of multilinguality and fall short of performance in monolingual settings by a large margin .
Revised:  We benchmark a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods based on multilingual pretraining and zero-shot fine-tuning transfer suffer from the curse of multilinguality and fall short of performance in monolingual settings by a large margin .
Edit intent: clarity
Original:  We benchmark a range of state-of-the-art models on this novel dataset, revealing that  current methods based on multilingual pretraining and zero-shot fine-tuning transfer suffer from the curse of multilinguality and fall short of performance in monolingual settings by a large margin .
Revised:  We benchmark a range of state-of-the-art models on this novel dataset, revealing that current methods based on multilingual pretraining and zero-shot fine-tuning falls short compared to translation-based transfer .
Edit intent: clarity
Original:  Previous work has aligned sentences from original and simplified corpora such as English Wikipedia and Simple English Wikipedia, but this limits corpus size, domain, and language. In this work, we propose using unsupervised mining techniques to automatically create training corpora for simplification in multiple languages from raw Common Crawl web data . When coupled with a controllable generation mechanism that can flexibly adjust attributes such as length and lexical complexity , these mined paraphrase corpora can be used to train simplification systems in any language .
Revised:  We introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that does not require labeled simplification data. MUSS uses a novel approach to sentence simplification that trains strong models using sentence-level paraphrase data instead of proper simplification data. These models leverage unsupervised pretraining and controllable generation mechanisms to flexibly adjust attributes such as length and lexical complexity , these mined paraphrase corpora can be used to train simplification systems in any language .
Edit intent: meaning-changed
Original:  We evaluate our approach on English, French, and Spanish simplification benchmarks and reach state-of-the-art performance with a totally unsupervised approach. We will release our models and code to mine the data in any language included in Common Crawl .
Revised:  We evaluate our approach on English, French, and Spanish simplification benchmarks and closely match or outperform the previous best supervised results, despite not using any labeled simplification data. We push the state of the art further by incorporating labeled simplification data .
Edit intent: meaning-changed
Original:  Overall, we identify six elements that are potentially necessary for BERT to be multilingual. Architectural factors that contribute to multilinguality are underparameterization, shared special tokens (e.g., " CLS%DIFDELCMD < ]%%% "), shared position embeddings and replacing masked tokens with random tokens.
Revised:  Overall, we identify CLS%DIFDELCMD < ]%%% "), shared position embeddings and replacing masked tokens with random tokens.
Edit intent: coherence
Original:  Architectural factors that contribute to multilinguality are underparameterization, shared special tokens (e.g., " CLS%DIFDELCMD < ]%%% "), shared position embeddings and replacing masked tokens with random tokens.
Revised:  Architectural factors that contribute to multilinguality are underparameterization, shared special tokens (e.g., " %DIFDELCMD < ]%%% "), shared position embeddings and replacing masked tokens with random tokens.
Edit intent: fluency
Original:  Architectural factors that contribute to multilinguality are underparameterization, shared special tokens (e.g., " CLS%DIFDELCMD < ]%%% "), shared position embeddings and replacing masked tokens with random tokens. Factors related to training data that are beneficial for multilinguality are similar word order and comparability of corpora .
Revised:  Architectural factors that contribute to multilinguality are underparameterization, shared special tokens (e.g., " CLS%DIFDELCMD < ]%%% four architectural and two linguistic elements that influence multilinguality. Based on our insights, we experiment with a multilingual pretraining setup that modifies the masking strategy using VecMap, i.e., unsupervised embedding alignment. Experiments on XNLI with three languages indicate that our findings transfer from our small setup to larger scale settings .
Edit intent: meaning-changed
Original:  We present our work as an example of applied visual analytics research, describing the design iterations that allowed us to move from experimentation to production, as well as the perspectives of the research team and the customer-facing team (Microsoft Workplace Analytics) at each stage in this process.
Revised:  We present our work as an example of applied visual analytics research, describing the design iterations that allowed us to move from experimentation to production, as well as the perspectives of the research team and the customer-facing team at each stage in this process.
Edit intent: coherence
Original:  In this paper, we leverage the differential privacy (DP) framework , to formulate and enforce rigorous privacy constraints on deep graph generation models, with a focus on edge-DP to guarantee individual link privacy.
Revised:  In this paper, we leverage the differential privacy (DP) framework to formulate and enforce rigorous privacy constraints on deep graph generation models, with a focus on edge-DP to guarantee individual link privacy.
Edit intent: fluency
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting proper noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: meaning-changed
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction-based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: fluency
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, while ensuring data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: coherence
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph discrimination . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: clarity
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGAN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: clarity
Original:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate networks with effectively preserved global structure and rigorously protected individual link privacy.
Revised:  In particular, we enforce edge-DP by injecting Gaussian noise to the gradients of a link reconstruction based graph generation model, and ensure data utility by improving structure learning with structure-oriented graph comparison . Extensive experiments on two real-world network datasets show that our proposed DPGGEN model is able to generate graphs with effectively preserved global structure and rigorously protected individual link privacy.
Edit intent: clarity
Original:  For this, we present MedType, a novel method that leverages the surrounding context to identify the semantic type of a mentionand uses it for filtering out candidate concepts of the wrong types.
Revised:  For this, we present MedType, a fully modular system that prunes out irrelevant candidate concepts based on the predicted semantic type of a mentionand uses it for filtering out candidate concepts of the wrong types.
Edit intent: meaning-changed
Original:  For this, we present MedType, a novel method that leverages the surrounding context to identify the semantic type of a mentionand uses it for filtering out candidate concepts of the wrong types. We further present two novel largescale, automatically-created datasetsof medical entity mentions: WIKIMED, a Wikipediabased dataset for cross-domain transfer learning, and PUBMEDDS, a distantly-supervised dataset of medical entity mentions in biomedical abstracts. Through extensive experiments across several datasetsand methods, we demonstrate that MedType pre-trained on our proposed datasets substantially improve medical entity linking and gives state-of-the-art performance.
Revised:  For this, we present MedType, a novel method that leverages the surrounding context to identify the semantic type of an entity mention. We incorporate MedType into five off-the-shelf toolkits for medical entity linking and demonstrate that it consistently improves entity linking performance across several benchmark datasets. To address the dearth of annotated training data for medical entity linking, we present WikiMed and PubMedDS, two large-scale medical entity linking datasets, and demonstrate that pre-training MedType on these datasets further improves entity linking performance.
Edit intent: clarity