Before revision: We present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model finetuning.
After revision: Language model fine-tuning is essential for modern natural language processing, but is computationally expensive and time-consuming. Further, the effectiveness of fine-tuning is limited by the inclusion of training examples that negatively affect performance. Here we present a general finetuning meta-method that we call information gain filtration for improving the overall training efficiency and final performance of language model finetuning.