  The recognition of personalized content, such as contact names, remains a
challenging problem for end-to-end speech recognition systems. In this work, we
demonstrate how first and second-pass rescoring strategies can be leveraged
together to improve the recognition of such words. Following previous work, we
use a shallow fusion approach to bias towards recognition of personalized
content in the first-pass decoding. We show that such an approach can improve
personalized content recognition by up to 16% with minimum degradation on the
general use case. We describe a fast and scalable algorithm that enables our
biasing models to remain at the word-level, while applying the biasing at the
subword level. This has the advantage of not requiring the biasing models to be
dependent on any subword symbol table. We also describe a novel second-pass
de-biasing approach: used in conjunction with a first-pass shallow fusion that
optimizes on oracle WER, we can achieve an additional 14% improvement on
personalized content recognition, and even improve accuracy for the general use
case by up to 2.5%.
