  Meta-learning models transfer the knowledge acquired from previous tasks to
quickly learn new ones. They are tested on benchmarks with a fixed number of
data points per training task. This number is usually arbitrary and it is
unknown how it affects the performance. Since labelling of data is expensive,
finding the optimal allocation of labels across training tasks may reduce
costs: given a fixed budget of labels, should we use a small number of highly
labelled tasks, or many tasks with few labels each? We show that: 1) The
optimal number of data points per task depends on the budget, but it converges
to a unique constant value for large budgets; 2) Convergence occurs around the
interpolation threshold of the model. We prove our results mathematically on
mixed linear regression, and we show empirically that the same results hold for
nonlinear regression and few-shot image classification on CIFAR-FS and
mini-ImageNet. Our results suggest a simple and efficient procedure for data
collection: the optimal allocation of data can be computed at low cost, by
using relatively small data, and collection of additional data can be optimized
by the knowledge of the optimal allocation.
