Gradient Ascent Post-training Enhances Language Model Generalization (2023)

Abstract: In this work, we empirically show that updating pretrained LMs (350M, 1.3B, 2.7B) with just a few steps of Gradient Ascent Post-training (GAP) on random, unlabeled text corpora enhances its zero-shot generalization capabilities across diverse NLP tasks. Specifically, we show that GAP can allow LMs to become comparable to 2-3x times larger LMs across 12 different NLP tasks. We also show that applying GAP on out-of-distribution corpora leads to the most reliable performance improvements. Our findings indicate that GAP can be a promising method for improving the generalization capability of LMs without any task-specific fine-tuning.

## introduction recently, language models (lms) pretrained on a vast amount of text corpora have shown to be capable of performing diverse downstream nlp tasks in a zero-shot manner @xcite @xcite or through in-context learning @xcite without any gradient updates. this paradigm has been preferred over task-specific fine-tuning @xcite , which requires considerable amount of labeled data for the given target task. motivated by the positive effect of gradient ascent during fine-tuning @xcite , in this work, we explore whether adapting pretrained lms with gradient ascent post-training (gap) on random, unlabeled text corpora can bring any benefits in terms of enhancing its generalization capabilities of performing diverse downstream nlp tasks in a zero-shot or few-shot manner without the need for task-specific training data. specifically, we apply just a few steps of gradient ascent to opt lms @xcite using for reference we also show the performance of 6.7b-opt baseline with a solid line. our main contributions can be summarized into two folds: while sam was proposed as a robust fine-tuning methodology that targets convergence on supervised dataset, we instead explore the benefits gradient ascent can bring without task-specific labeled data for generic lms. ## gradient ascent post-training (gap) in this section, we give a formal definition of gap. specifically, given an lm with parameters w and a sequence of tokens @xmath0, ..., x n ), gap is defined as: ## conclusion in this work, we introduce gap, a novel method of improving the generalization capability of lms without any task-specifc data by sampling random text and performing gradient ascent for a few steps. we show that our approach is (1) simple to use, (2) effective in making more robust lms, and (3) has much room for improvements for future work when scaling the number of gap runs (e.g. >300) and choosing specific text samples (e.g. out-ofdistribution text) to perform gap on. thus, we urge the community to consider gap when prompting off-the-shelf pretrained lms for performing diverse downstream nlp tasks.