Listen, Understand and Translate: Triple Supervision Decouples End-to-end Speech-to-text Translation
Journal: Proceedings of the AAAI Conference on Artificial Intelligence
Year: 2021

An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel corpus. Can we build a system to fully utilize signals in a parallel ST corpus? We are inspired by human understanding system which is composed of auditory perception and cognitive processing. In this paper, we propose Listen-Understand-Translate, (LUT), a unified framework with triple supervision signals to decouple the