  Existing approaches to few-shot learning deal with tasks that have
persistent, rigid notions of classes. Typically, the learner observes data only
from a fixed number of classes at training time and is asked to generalize to a
new set of classes at test time. Two examples from the same class would always
be assigned the same labels in any episode. In this work, we consider a
realistic setting where the similarities between examples can change from
episode to episode depending on the task context, which is not given to the
learner. We define new benchmark datasets for this flexible few-shot scenario,
where the tasks are based on images of faces (Celeb-A), shoes (Zappos50K), and
general objects (ImageNet-with-Attributes). While classification baselines and
episodic approaches learn representations that work well for standard few-shot
learning, they suffer in our flexible tasks as novel similarity definitions
arise during testing. We propose to build upon recent contrastive unsupervised
learning techniques and use a combination of instance and class invariance
learning, aiming to obtain general and flexible features. We find that our
approach performs strongly on our new flexible few-shot learning benchmarks,
demonstrating that unsupervised learning obtains more generalizable
representations.
