  Intent classification is a task in spoken language understanding. An intent
classification system is usually implemented as a pipeline process, with a
speech recognition module followed by text processing that classifies the
intents. There are also studies of end-to-end system that takes acoustic
features as input and classifies the intents directly. Such systems don't take
advantage of relevant linguistic information, and suffer from limited training
data. In this work, we propose a novel intent classification framework that
employs acoustic features extracted from a pretrained speech recognition system
and linguistic features learned from a pretrained language model. We use
knowledge distillation technique to map the acoustic embeddings towards
linguistic embeddings. We perform fusion of both acoustic and linguistic
embeddings through cross-attention approach to classify intents. With the
proposed method, we achieve 90.86% and 99.07% accuracy on ATIS and Fluent
speech corpus, respectively.
