  Few-shot learning arises in important practical scenarios, such as when a
natural language understanding system needs to learn new semantic labels for an
emerging, resource-scarce domain. In this paper, we explore retrieval-based
methods for intent classification and slot filling tasks in few-shot settings.
Retrieval-based methods make predictions based on labeled examples in the
retrieval index that are similar to the input, and thus can adapt to new
domains simply by changing the index without having to retrain the model.
However, it is non-trivial to apply such methods on tasks with a complex label
space like slot filling. To this end, we propose a span-level retrieval method
that learns similar contextualized representations for spans with the same
label via a novel batch-softmax objective. At inference time, we use the labels
of the retrieved spans to construct the final structure with the highest
aggregated score. Our method outperforms previous systems in various few-shot
settings on the CLINC and SNIPS benchmarks.
