The support/query (S/Q) training protocol is widely used in meta-learning. S/Q protocol trains a task-specific model on S and then evaluates it on Q to optimize the meta-model using query loss , which depends on size and quality of Q. In this paper, we study a new S / T protocolfor meta-learning. Assuming that we have access to the theoretically optimal model T for a task, we can directly match the task-specific model trained on S to T. S / T protocol offers a more accurate evaluation since it does not rely on possibly biased and noisy query instances. There are two challenges in putting S / T protocol into practice. Firstly, we have to determine how to match the task-specific model to T. To this end, we minimize the discrepancy between them on a fictitious dataset generated by adversarial learning, and distill the prediction ability of T to the task-specific model. Secondly, we usually do not have ready-made optimal models. As an alternative, we construct surrogate/\mathcal{T} protocol in a typical application of meta-learning, \mathit{i.e.}, few-shot learning. In detail, after constructing } target models by fine-tuning on local tasks the globally pre-trained meta-model, maintaining both efficiency and veracity .
Meta-learning can extract an inductive bias from previous learning experience and assist the training processes of new tasks. It is often realized through optimizing a meta-model with the evaluation loss of a series of task-specific solvers. Most existing algorithms sample non-overlapping \mathit{support / \mathcal{Q / \mathcal{T / \mathcal{T into practice. We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources. Furthermore, we empirically verify the effectiveness of \mathcal{S/\mathcal{T} protocol in a typical application of meta-learning, \mathit{i.e.}, few-shot learning. In detail, after constructing } target models by fine-tuning the pre-trained network on those hard tasks, we match the task-specific solvers to target models via knowledge distillation. Experiments demonstrate the superiority of our proposal .
R
The support/query (S/Q) training protocol is widely used in meta-learning. S/Q protocol trains a task-specific model on S and then evaluates it on Q to optimize the
Meta-learning can extract an inductive bias from previous learning experience and assist the training processes of new tasks. It is often realized through optimizing a
R
using query loss , which depends on size and quality of Q. In this paper, we study a new S
with the evaluation loss of a series of task-specific solvers. Most existing algorithms sample non-overlapping \mathit{support
R
T protocolfor meta-learning. Assuming that we have access to the theoretically optimal model T for a task, we can directly match the task-specific model trained on S to T. S
\mathcal{Q
R
T protocol offers a more accurate evaluation since it does not rely on possibly biased and noisy query instances. There are two challenges in putting S
\mathcal{T
R
T protocol
\mathcal{T
R
Firstly, we have to determine how to match the task-specific model to T. To this end, we minimize the discrepancy between them on a fictitious dataset generated by adversarial learning, and distill the prediction ability of T to the task-specific model. Secondly, we usually do not have ready-made optimal models. As an alternative, we construct surrogate
We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources. Furthermore, we empirically verify the effectiveness of \mathcal{S
R
on local tasks the globally
the
R
meta-model, maintaining both efficiency and veracity
network on those hard tasks, we match the task-specific solvers to target models via knowledge distillation. Experiments demonstrate the superiority of our proposal