Q: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Strategy-based methods depart from the pre-training stage, seeking to take advantage of the pre-trained models during the process of target task learning. The approaches include fine-tuning schedules that elaborately design the control of learning rates for optimization, proxy tasks that leverage labeled data to help the pre-trained model better fit the target data distribution, and knowledge distillation approaches that ditch the paradigm of initialization with pre-trained parameters by adopting the pre-trained model as a teacher network.
Question: How strategy-based methods handle obstacles in NLG?
A:
fine-tuning schedules that elaborately design the control of learning rates for optimization proxy tasks that leverage labeled data to help the pre-trained model better fit the target data distribution knowledge distillation approaches that ditch the paradigm of initialization with pre-trained parameters by adopting the pre-trained model as a teacher network