  Slot filling is identifying contiguous spans of words in an utterance that
correspond to certain parameters (i.e., slots) of a user request/query. Slot
filling is one of the most important challenges in modern task-oriented dialog
systems. Supervised learning approaches have proven effective at tackling this
challenge, but they need a significant amount of labeled training data in a
given domain. However, new domains (i.e., unseen in training) may emerge after
deployment. Thus, it is imperative that these models seamlessly adapt and fill
slots from both seen and unseen domains -- unseen domains contain unseen slot
types with no training data, and even seen slots in unseen domains are
typically presented in different contexts. This setting is commonly referred to
as zero-shot slot filling. Little work has focused on this setting, with
limited experimental evaluation. Existing models that mainly rely on
context-independent embedding-based similarity measures fail to detect slot
values in unseen domains or do so only partially. We propose a new zero-shot
slot filling neural model, LEONA, which works in three steps. Step one acquires
domain-oblivious, context-aware representations of the utterance word by
exploiting (a) linguistic features; (b) named entity recognition cues; (c)
contextual embeddings from pre-trained language models. Step two fine-tunes
these rich representations and produces slot-independent tags for each word.
Step three exploits generalizable context-aware utterance-slot similarity
features at the word level, uses slot-independent tags, and contextualizes them
to produce slot-specific predictions for each word. Our thorough evaluation on
four diverse public datasets demonstrates that our approach consistently
outperforms the SOTA models by 17.52%, 22.15%, 17.42%, and 17.95% on average
for unseen domains on SNIPS, ATIS, MultiWOZ, and SGD datasets, respectively.
