PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models
Journal: arXiv (Cornell University)
Year: 2024

Recently, there have been attempts to integrate various speech processing tasks into a unified model. However, few previous works directly demonstrated that joint optimization of diverse tasks in multitask speech models has positive influence on the performance of individual tasks. In this paper we present a multitask speech model -- PolySpeech, which supports speech recognition, speech synthesis, and two speech classification tasks. PolySpeech takes multi-modal language model as its core struct