  In this paper, we look at the problem of few-shot classification that aims to
learn a classifier for previously unseen classes and domains from few labeled
samples. Recent methods use adaptation networks for aligning their features to
new domains or select the relevant features from multiple domain-specific
feature extractors. In this work, we propose to learn a single set of universal
deep representations by distilling knowledge of multiple separately trained
networks after co-aligning their features with the help of adapters and
centered kernel alignment. We show that the universal representations can be
further refined for previously unseen domains by an efficient adaptation step
in a similar spirit to distance learning methods. We rigorously evaluate our
model in the recent Meta-Dataset benchmark and demonstrate that it
significantly outperforms the previous methods while being more efficient. Our
code will be available at https://github.com/VICO-UoE/URL.
