Text-only Domain Adaptation using Unified Speech-Text Representation in
  Transducer
Authors: name: Lu Huang; name: Boyu Li; name: Jun Zhang; name: Lu Lu; name: Zejun Ma
Published: 2023-06-07 00:33:02

Abstract: Domain adaptation using text-only corpus is challenging in end-to-end(E2E)
speech recognition. Adaptation by synthesizing audio from text through TTS is
resource-consuming. We present a method to learn Unified Speech-Text
Representation in Conformer Transducer(USTR-CT) to enable fast domain
adaptation using the text-only corpus. Different from the previous textogram
method, an extra text encoder is introduced in our work to learn text
representation and is removed during inference, so there is no modification for
online deployment. To improve the efficiency of adaptation, single-step and
multi-step adaptations are also explored. The experiments on adapting
LibriSpeech to SPGISpeech show the proposed method reduces the word error
rate(WER) by relatively 44% on the target domain, which is better than those of
TTS method and textogram method. Also, it is shown the proposed method can be
combined with internal language model estimation(ILME) to further improve the
performance.