Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation
Journal: Zurich Open Repository and Archive (University of Zurich)
Year: 2023

For end-to-end speech translation, regularizing the encoder with the Connectionist Temporal Classification (CTC) objective using the source transcript or target translation as labels can greatly improve quality. However, CTC demands an extra prediction layer over the vocabulary space, bringing in non-negligible model parameters and computational overheads, although this layer becomes useless at inference. In this paper, we re-examine the need for genuine vocabulary labels for CTC for regularizat