wanyu/IteraTeR_full_sent huggingface dataset:
Original:  It gains 4.7\% accuracy over the best of both models on the computation and language domain and loses 2.4\% against the best of both on the machine learning domain . Compared to plain HANs, accuracy increases on both domains, with 1.5\% and 2\% respectively .
Revised:  It gains 4.7\% accuracy over the best of both models on the computation and language domain and loses 2.4\% against the best of both on the machine learning domain . Compared to plain HANs, accuracy increases on all three domains. On the computation and language domain our new model works best overall, and increases accuracy 4.7\% over the best literature result .
Edit intent: meaning-changed
Original:  For our HAN-system with structure-tags we reach 28.5\% explained variance, an improvement of  1.0\% over HANs without structure-tags .
Revised:  For our HAN-system with structure-tags we reach 28.5\% explained variance, an improvement of 1.8\% over our reimplementation of the BiLSTM-based model as well as 1.0\% over HANs without structure-tags .
Edit intent: meaning-changed
Original:  Traditional multi-task  learning models implicitly capture the correlations between NER and RE. However, there exist intrinsic connections between the output of NER and RE. In this study, we argue that an explicit interaction between the NER model and the RE model will better guide the training of both models. Based on the traditional multi-task learning framework, we design an interactive feature encoding method to capture the intrinsic connections between NER and RE tasks.
Revised:  Traditional multi-task learning techniques to address the problem learn interactions among the two tasks through a shared network, where the shared information is passed into the task-specific networks for prediction. However, such an approach hinders the model from learning explicit interactions between the two tasks to improve the performance on the individual tasks. As a solution, we design a multi-task learning framework, we design an interactive feature encoding method to capture the intrinsic connections between NER and RE tasks.
Edit intent: meaning-changed
Original:  Based on the traditional multi-task learning framework, we design an interactive feature encoding method to capture the intrinsic connections between NER and RE tasks. In addition, we propose a recurrent interaction network to progressively capture the correlation between the two models . Empirical studies on two real-world datasets confirm the superiority of the proposed model.
Revised:  Based on the traditional multi-task learning model which we refer to as recurrent interaction network to progressively capture the correlation between the two models . Empirical studies on two real-world datasets confirm the superiority of the proposed model.
Edit intent: clarity
Original:  In addition, we propose a recurrent interaction network to progressively capture the correlation between the two models . Empirical studies on two real-world datasets confirm the superiority of the proposed model.
Revised:  In addition, we propose a recurrent interaction network which allows the learning of interactions dynamically, to effectively model task-specific features for classification . Empirical studies on two real-world datasets confirm the superiority of the proposed model.
Edit intent: meaning-changed
Original:  Therefore, we propose a new model, JEANS , which jointly represents multilingual KGs and text corpora in a shared embedding scheme, and seeks to improve entity alignment with incidental supervision signals from text.
Revised:  Therefore, we propose an incidentally supervised model, JEANS , which jointly represents multilingual KGs and text corpora in a shared embedding scheme, and seeks to improve entity alignment with incidental supervision signals from text.
Edit intent: clarity
Original:  Then, two learning processes are conducted: (i) an embedding learning process to encode the KG and text of each language in one embedding space, and (ii) a self-learning based alignment learning process to iteratively induce the correspondence of entities and that of lexemes between embeddings.
Revised:  Then, two learning processes are conducted: (i) an embedding learning process to encode the KG and text of each language in one embedding space, and (ii) a selflearning based alignment learning process to iteratively induce the correspondence of entities and that of lexemes between embeddings.
Edit intent: fluency
Original:  Then, two learning processes are conducted: (i) an embedding learning process to encode the KG and text of each language in one embedding space, and (ii) a self-learning based alignment learning process to iteratively induce the correspondence of entities and that of lexemes between embeddings.
Revised:  Then, two learning processes are conducted: (i) an embedding learning process to encode the KG and text of each language in one embedding space, and (ii) a self-learning based alignment learning process to iteratively induce the matching of entities and that of lexemes between embeddings.
Edit intent: clarity
Original:  Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard of Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts that relate to their existing knowledge .
Revised:  Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard-of-Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts that relate to their existing knowledge .
Edit intent: fluency
Original:  Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard of Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts that relate to their existing knowledge .
Revised:  Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard of Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts related to what they know .
Edit intent: clarity
Original:  This dataset is annotated with pre-existing user knowledge, message-level dialog acts, message grounding to Wikipedia,  user reactions to messages , and per-dialog ratings. Our analysis shows that responses which incorporate a user's prior knowledge do increase engagement.
Revised:  This dataset is annotated with pre-existing user knowledge, message-level dialog acts, message grounding to Wikipedia, user reactions to messages . Responses using a user's prior knowledge do increase engagement.
Edit intent: clarity
Original: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Revised: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequences of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Edit intent: fluency
Original: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Revised: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to text processed by classifiers . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Edit intent: clarity
Original: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Revised: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Edit intent: clarity
Original: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Revised: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in such attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Edit intent: clarity
Original: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often unnatural, do not carry much semantic meaning, and can be easily distinguished from natural text.
Revised: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequence of words added to any input instance . Despite being highly successful, the word sequences produced in these attacks are often ungrammatical and can be easily distinguished from natural text.
Edit intent: clarity
Original:  To achieve this, we leverage an adversarially regularized autoencoder (ARAE) to generate triggers and propose a gradient-based search method to output natural text that fools a target classifier. Experiments on two different classification tasks demonstrate the effectiveness of our attacks while also being less identifiable than previous approaches on three simple detection metrics .
Revised:  To achieve this, we leverage an adversarially regularized autoencoder (ARAE) to generate triggers and propose a gradient-based search that aims to maximize the downstream classifier's prediction loss. Our attacks effectively reduce model accuracy on classification tasks while being less identifiable than previous approaches on three simple detection metrics .
Edit intent: clarity
Original:  Experiments on two different classification tasks demonstrate the effectiveness of our attacks while also being less identifiable than previous approaches on three simple detection metrics .
Revised:  Experiments on two different classification tasks demonstrate the effectiveness of our attacks while also being less identifiable than prior models as per automatic detection metrics and human-subject studies. Our aim is to demonstrate that adversarial attacks can be made harder to detect than previously thought and to enable the development of appropriate defenses .
Edit intent: meaning-changed
Original: We present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model finetuning.
Revised: Language model fine-tuning is essential for modern natural language processing, but is computationally expensive and time-consuming. Further, the effectiveness of fine-tuning is limited by the inclusion of training examples that negatively affect performance. Here we present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model finetuning.
Edit intent: coherence
Original: We present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model finetuning. This method uses a secondary learner which attempts to quantify the benefit of finetuning the language model on each given example. During the finetuning process, we use this learner to decide whether or not each given example should be trained on or skipped .
Revised: We present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model fine-tuning. We define the information gain of an example as the improvement on a test metric after training on that example. A secondary learner is then trained to approximate this quantity. During fine-tuning, this learner selects informative examples and skips uninformative ones .
Edit intent: meaning-changed
Original:  We show that it suffices for this learner to be simple and that the finetuning process itself is dominated by the relatively trivial relearning of a new unigram frequency distribution over the modelled language domain, a process which the learner aids. Our method trains to convergence using 40\% fewer batches than normal finetuning, and achieves a median perplexity of 54.0 on a books dataset compared to a median perplexity of 57.3 for standard finetuning using the same neural architecture .
Revised:  We show that our method has consistent improvement across datasets, fine-tuning tasks, and language model architectures. For example, we achieve a median perplexity of 54.0 on a books dataset compared to a median perplexity of 57.3 for standard finetuning using the same neural architecture .
Edit intent: clarity
Original:  Our method trains to convergence using 40\% fewer batches than normal finetuning, and achieves a median perplexity of 54.0 on a books dataset compared to a median perplexity of 57.3 for standard finetuning using the same neural architecture .
Revised:  Our method trains to convergence using 40\% fewer batches than normal finetuning, and achieves a median perplexity of 54.0 on a books dataset compared to 57.3 for standard finetuning using the same neural architecture .
Edit intent: fluency
Original:  Our method trains to convergence using 40\% fewer batches than normal finetuning, and achieves a median perplexity of 54.0 on a books dataset compared to a median perplexity of 57.3 for standard finetuning using the same neural architecture .
Revised:  Our method trains to convergence using 40\% fewer batches than normal finetuning, and achieves a median perplexity of 54.0 on a books dataset compared to a median perplexity of 57.3 for standard fine-tuning. We present statistical evidence that offers insight into the improvements of our method over standard fine-tuning. The generality of our method leads us to propose a new paradigm for language model fine-tuning -- we encourage researchers to release pretrained secondary learners on common corpora to promote efficient and effective fine-tuning, thereby improving the performance and reducing the overall energy footprint of language model fine-tuning .
Edit intent: meaning-changed
Original:  We establish new connections between the encoding dimension and the number of unique terms in each document and query, using both theoretical and empirical analysis. We show an upper bound on the encoding size, which may be unsustainably large for long documents. For cross-attention models, we show an upper bound using much smaller encodings per token, but such models are difficult to scale to realistic retrieval problems due to computational cost .
Revised:  We establish new connections between the encoding dimension , the margin between gold and lower-ranked documents, and the document length, suggesting limitations in the capacity of fixed-length encodings to support precise retrieval of long documents .
Edit intent: meaning-changed
Original:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of  attentional architectures, and explore a sparse-dense hybrid to capitalize on the precision of sparse retrieval.
Revised:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of more costly attentional architectures, and explore a sparse-dense hybrid to capitalize on the precision of sparse retrieval.
Edit intent: meaning-changed
Original:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of  attentional architectures, and explore a sparse-dense hybrid to capitalize on the precision of sparse retrieval.
Revised:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of attentional architectures, and explore sparse-dense hybrid to capitalize on the precision of sparse retrieval.
Edit intent: fluency
Original:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of  attentional architectures, and explore a sparse-dense hybrid to capitalize on the precision of sparse retrieval.
Revised:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of attentional architectures, and explore a sparse-dense hybrids to capitalize on the precision of sparse retrieval.
Edit intent: fluency
Original:  Moreover, there is a lack of benchmark datasets to evaluate the suitability of existing metrics in terms of correctness. To study a better metric for GenQA, we first create high-quality human judgments of correctness on two standard GenQA datasets. Using our human-evaluation datasets, we show that widely used n-gram similarity metrics do not correlate with human judgments .
Revised:  Especially, widely used n-gram similarity metrics do not correlate with human judgments .
Edit intent: coherence
Original: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Revised: We present HERO, a novel framework for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Edit intent: clarity
Original: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Revised: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language omni-representation learning . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Edit intent: clarity
Original: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Revised: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical structure , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Edit intent: clarity
Original: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local textual context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Revised: We present HERO, a Hierarchical EncodeR for Omni-representation learning, for large-scale video+language pre-training . HERO encodes multimodal inputs in a hierarchical fashion , where local context of a video frame is captured by a Cross-modal Transformer via multimodal fusion, and global video context is captured by a Temporal Transformer.
Edit intent: clarity
Original:  Besides standard Masked Language Modeling (MLM) and Masked Frame Modeling (MFM) objectives, we design two new pre-training tasks: (i) Video-Subtitle Matching (VSM), where the model predicts both global and local temporal alignment;
Revised:  In addition to standard Masked Language Modeling (MLM) and Masked Frame Modeling (MFM) objectives, we design two new pre-training tasks: (i) Video-Subtitle Matching (VSM), where the model predicts both global and local temporal alignment;
Edit intent: clarity
Original:  Different from previous work that mostly focused on cooking or narrated instructional videos, HERO is jointly trained on HowTo100M and large-scale TV show datasets to learn complex social scenes, dynamics backdrop transitions and multi-character interactions.
Revised:  Different from previous work that mostly focused on cooking or narrated instructional videos, HERO is jointly trained on HowTo100M and large-scale TV datasets to gain deep understanding of complex social dynamics with multi-character interactions.
Edit intent: clarity
Original:  Extensive experiments demonstrate that HERO achieves new state of the art on both text-based video moment retrieval and video question answering tasks across different domains  .
Revised:  Extensive experiments demonstrate that HERO achieves new state of the art on multiple benchmarks over Text-based Video/Video-moment Retrieval, Video Question Answering (QA), Video-and-language Inference and Video Captioning tasks across different domains .
Edit intent: meaning-changed
Original:  Extensive experiments demonstrate that HERO achieves new state of the art on both text-based video moment retrieval and video question answering tasks across different domains  .
Revised:  Extensive experiments demonstrate that HERO achieves new state of the art on both text-based video moment retrieval and video question answering tasks across different domains . We also introduce two new challenging benchmarks How2QA and How2R for Video QA and Retrieval, collected from diverse video content over multimodalities .
Edit intent: meaning-changed