Before revision: We study the problem of few-sample fine-tuning of BERT contextual representations, and identify three sub-optimal choices in current, broadly adopted practices.
After revision: This paper is a study of fine-tuning of BERT contextual representations, and identify three sub-optimal choices in current, broadly adopted practices.