wanyu/IteraTeR_full_sent huggingface dataset:
Original:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models  learn to encode linguistic properties even when not needed for a task .
Revised:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case study in Natural Language Inference, showing that models learn to encode linguistic properties even when not needed for a task .
Edit intent: fluency
Original:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models  learn to encode linguistic properties even when not needed for a task .
Revised:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models can learn to encode linguistic properties even when not needed for a task .
Edit intent: clarity
Original:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models  learn to encode linguistic properties even when not needed for a task .
Revised:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models learn to encode linguistic properties even if they are not needed for a task .
Edit intent: clarity
Original:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models  learn to encode linguistic properties even when not needed for a task . We identify that pre-trained word embeddings play a considerable role in encoding these properties rather than the training task itself, highlighting the importance of careful controls when designing probing experiments.
Revised:  However, to what extent is the information encoded in a sentence representation actually used for the task which the encoder is trained on ? In this work, we examine this probing paradigm through a case-study in Natural Language Inference, showing that models learn to encode linguistic properties even when not needed for the task on which the model was trained. We further identify that pretrained word embeddings play a considerable role in encoding these properties rather than the training task itself, highlighting the importance of careful controls when designing probing experiments.
Edit intent: clarity
Original: Fluent communication requires understanding your audience. In the new collaborative task of Vision-and-Dialog Navigation, one agent must ask questions and follow instructive answers, while the other must provide those answers. We introduce the first true dialog navigation agents in the literature which generate full conversations, and introduce the Recursive Mental Model (RMM) to conduct these dialogs.
Revised: Language-guided robots must be able to both ask humans questions and understand answers. Much existing work focuses only on the latter. In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers. Inspired by theory of mind, we propose the Recursive Mental Model (RMM) to conduct these dialogs.
Edit intent: coherence
Original:  We introduce the first true dialog navigation agents in the literature which generate full conversations, and introduce the Recursive Mental Model (RMM) to conduct these dialogs. RMM dramatically improves generated language questionsand answers by recursively propagating reward signals to find the question expected to elicit the best answer, and the answer expected to elicit the best navigation . Additionally, we provide baselines for future work to build on when investigating the unique challenges of embodied visual agents that not only interpret instructions but also ask questions in natural language .
Revised:  We introduce the first true dialog navigation agents in the literature which generate full conversations, and introduce the Recursive Mental Model (RMM) . The navigating agent models the guiding agent to simulate answers given candidate generated questions in natural language .
Edit intent: clarity
Original:  Additionally, we provide baselines for future work to build on when investigating the unique challenges of embodied visual agents that not only interpret instructions but also ask questions in natural language .
Revised:  Additionally, we provide baselines for future work to build on when investigating the unique challenges of embodied visual agents that not only interpret instructions but also ask questions . The guiding agent in turn models the navigating agent to simulate navigation steps it would take to generate answers. We use the progress agents make towards the goal as a reinforcement learning reward signal to directly inform not only navigation actions, but also both question and answer generation. We demonstrate that RMM enables better generalization to novel environments. Interlocutor modelling may be a way forward for human-agent dialogue where robots need to both ask and answer questions .
Edit intent: meaning-changed
Original:  Via extensive experiments, we find that (1) random alignment matrices surprisingly perform quite competitively and (2) learning attention weights from token-token (query-key) interactions is  not that important after all.
Revised:  Via extensive experiments, we find that (1) random alignment matrices surprisingly perform quite competitively and (2) learning attention weights from token-token (query-key) interactions is useful but not that important after all.
Edit intent: meaning-changed
Original:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Revised:  Synthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Edit intent: clarity
Original:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Revised:  Our experimental results show that is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Edit intent: clarity
Original:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Revised:  Our experimental results show thatSynthesizer In our experiments, we first show that simple Synthesizers achieve highly competitive performance when compared against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Edit intent: meaning-changed
Original:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
Revised:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including machine translation, language modeling, text generation and GLUE/SuperGLUE benchmarks. When composed with dot product attention, we find that Synthesizers consistently outperform Transformers. Moreover, we conduct additional comparisons of Synthesizers against Dynamic Convolutions, showing that simple Random Synthesizer is not only 60
Edit intent: meaning-changed
Original:  By solving the differential equation system numerically, we can determine: -- the expected the number of informed nodes as a function time, -- the degree distribution of the set of informed nodes at every specific instant -- and the relationship between the node degree and its reception time.
Revised:  By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function time, -- the degree distribution of the set of informed nodes at every specific instant -- and the relationship between the node degree and its reception time.
Edit intent: fluency
Original:  By solving the differential equation system numerically, we can determine: -- the expected the number of informed nodes as a function time, -- the degree distribution of the set of informed nodes at every specific instant -- and the relationship between the node degree and its reception time.
Revised:  By solving the differential equation system numerically, we can determine: -- the expected the number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant -- and the relationship between the node degree and its reception time.
Edit intent: fluency
Original:  By solving the differential equation system numerically, we can determine: -- the expected the number of informed nodes as a function time, -- the degree distribution of the set of informed nodes at every specific instant -- and the relationship between the node degree and its reception time.
Revised:  By solving the differential equation system numerically, we can determine: -- the expected the number of informed nodes as a function time, -- the degree distribution of the set of informed nodes at every specific instant - and the relationship between the node degree and its reception time.
Edit intent: fluency
Original:  A probabilistic differential equation system is derived and solved numerically for the expected reception times. By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant - and the relationship between the node degree and its reception time.
Revised:  A probabilistic differential equation system . The numerical solution of these equations leads to the expected number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant - and the relationship between the node degree and its reception time.
Edit intent: clarity
Original:  By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant - and the relationship between the node degree and its reception time.
Revised:  By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function of time and reveals the relationship between the node degree and its reception time.
Edit intent: clarity
Original:  By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant - and the relationship between the node degree and its reception time.
Revised:  By solving the differential equation system numerically, we can determine: - the expected number of informed nodes as a function time, - the degree distribution of the set of informed nodes at every specific instant - and the relationship between the degrees of the nodes and their reception time.
Edit intent: clarity
Original:  The validity and accuracy of the model are justified by Monte Carlo network simulations through the analysis of information propagation in scale-free and Erdos-Renyi  networks.
Revised:  The validity and accuracy of the model are justified by Monte Carlo network simulations through the analysis of information propagation in scale-free and Erdos-Renyi networks. It has been found that the developed model provides more accurate results compared to the widely used network-based SI mean-field model, especially in sparse networks.
Edit intent: meaning-changed
Original:  To improve the recall of facts stated in the trainingtext , we combine BERT (Devlin et al., 2019) with a  kNN search over a large corpus .
Revised:  To improve the recall of facts stated in the trainingtext , we combine BERT (Devlin et al., 2019) with a traditional information retrieval step (IR) and a kNN search over a large corpus .
Edit intent: meaning-changed
Original:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ")  .
Revised:  Our contributions are as follows : i) BERT-kNN outperforms BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ") .
Edit intent: meaning-changed
Original:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ")  .
Revised:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., US city), but only kNN recovers the factually correct answer (e.g., " Vienna ") .
Edit intent: meaning-changed
Original:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ")  .
Revised:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Miami ") .
Edit intent: meaning-changed
Original:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ")  .
Revised:  Our contributions are as follows . i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., " Vienna ") . iii) Compared to BERT, BERT-kNN excels for rare facts. iv) BERT-kNN can easily handle facts not covered by BERT's training set, e.g., recent events .
Edit intent: meaning-changed
Original: Pre-trained language models (PTLM) have greatly improved performance on commonsense inference benchmarks, however, it remains unclear whether they share a human's ability to consistently make correct inferences under perturbations.
Revised: Pre-trained language models (PTLM) have impressive performance on commonsense inference benchmarks, however, it remains unclear whether they share a human's ability to consistently make correct inferences under perturbations.
Edit intent: clarity
Original: Pre-trained language models (PTLM) have greatly improved performance on commonsense inference benchmarks, however, it remains unclear whether they share a human's ability to consistently make correct inferences under perturbations. Prior studies of PTLMs have found inference deficits, but have failed to provide a systematic means of understanding whether these deficits are due to low inference abilities or poor inference robustness .
Revised: Pre-trained language models (PTLM) have greatly improved performance on commonsense inference benchmarks, but their ability to practically employ commonsense to communicate with humans is fiercely debated. Prior evaluations of PTLMs have found inference deficits, but have failed to provide a systematic means of understanding whether these deficits are due to low inference abilities or poor inference robustness .
Edit intent: clarity
Original:  Prior studies of PTLMs have found inference deficits, but have failed to provide a systematic means of understanding whether these deficits are due to low inference abilities or poor inference robustness . In this work, we address this gap by developing a procedure that allows for the systematized probing of both PTLMs' inference abilities and robustness. Our procedure centers around the methodical creation of logically-equivalent, but syntactically-different sets of probes, of which we create a corpus of 14,400 probes coming from 60 logically-equivalent sets that can be used to probe PTLMs in three task settings.
Revised:  Prior studies of PTLMs have focused on factual world knowledge or the ability to reason when the necessary knowledge is provided explicitly. However, effective communication with humans requires inferences based on implicit commonsense relationships, and robustness despite paraphrasing. In the pursuit of advancing fluid human-AI communication, we propose a new challenge, RICA, that evaluates the capabilities of making commonsense inferences and the robustness of these inferences to language variations. In our work, we develop a systematic procedure to probe PTLMs in three task settings.
Edit intent: meaning-changed
Original:  Our procedure centers around the methodical creation of logically-equivalent, but syntactically-different sets of probes, of which we create a corpus of 14,400 probes coming from 60 logically-equivalent sets that can be used to probe PTLMs in three task settings. We find that despite the recent success of large PTLMs on commonsense benchmarks, their performances on our probes are no better than random guessing (even with fine-tuning) and are heavily dependent on biases--the poor overall performance, unfortunately, inhibits us from studying robustness.
Revised:  Our procedure centers around the methodical creation of logically-equivalent, but syntactically-different sets of probes, of which we create a corpus of 14,400 probes coming from 60 logically-equivalent sets that can be used to probe PTLMs across three different evaluation settings. Extensive experiments on our generated probe sets show that PTLMs perform no better than random guessing (even with fine-tuning) and are heavily dependent on biases--the poor overall performance, unfortunately, inhibits us from studying robustness.
Edit intent: clarity
Original:  We find that despite the recent success of large PTLMs on commonsense benchmarks, their performances on our probes are no better than random guessing (even with fine-tuning) and are heavily dependent on biases--the poor overall performance, unfortunately, inhibits us from studying robustness. We hope our approach and initial probe set will assist future work in improving PTLMs' inference abilities , while also providing a probing set to test robustness under several linguistic variations--code and data will be released .
Revised:  We find that despite the recent success of large PTLMs on commonsense benchmarks, their performances on our probes are no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks. Our framework and probe sets can help future work improve PTLMs' inference abilities , while also providing a probing set to test robustness under several linguistic variations--code and data will be released .
Edit intent: clarity
Original: Pre-trained language models ( PTLM) have impressive performance on commonsense inference benchmarks, but their ability to practically employ commonsense to communicate with humans is fiercely debated. Prior evaluations of PTLMs have focused on factual world knowledge or the ability to reason when the necessary knowledge is provided explicitly. However, effective communication with humansrequires inferences based on implicit commonsense relationships, and robustness despite paraphrasing .
Revised: Pre-trained language models ( PTLM) have impressive performance on commonsense inference benchmarks, but their ability to practically employ commonsense to make robust inferences, which is crucial for effective communications with humans, is debated .
Edit intent: clarity
Original:  In the pursuit of advancing fluid human-AI communication, we propose a new challenge, RICA  , that evaluates the capabilities of making commonsense inferences and the robustness of these inferences to language variations.
Revised:  In the pursuit of advancing fluid human-AI communication, we propose a new challenge, RICA : Robust Inference capability based on Commonsense Axioms , that evaluates the capabilities of making commonsense inferences and the robustness of these inferences to language variations.
Edit intent: meaning-changed
Original:  In the pursuit of advancing fluid human-AI communication, we propose a new challenge, RICA  , that evaluates the capabilities of making commonsense inferences and the robustness of these inferences to language variations. In our work , we develop a systematic procedure to probe PTLMs across three different evaluation settings.
Revised:  In the pursuit of advancing fluid human-AI communication, we propose a new challenge, RICA , that evaluates robust commonsense inference despite textual perturbations. To generate data for this challenge , we develop a systematic procedure to probe PTLMs across three different evaluation settings.
Edit intent: clarity
Original:  Extensive experiments on our generated probe sets  show that PTLMs perform no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks.
Revised:  Extensive experiments on our generated probe sets with more than 10k statements show that PTLMs perform no better than random guessing (even with fine-tuning) , are heavily impacted by statistical biases, and are not robust to perturbation attacks.
Edit intent: meaning-changed