Before revision:  In addition to the target language sentence translation loss, LUT includes two auxiliary supervising signals to guide the acoustic encoder to extracts acoustic features from the input, and the semantic encoder to extract semantic features relevant to the source transcription text. We do experiments on English-French,  English-German and  English-Chinese speech translation benchmarks and the results demonstrate the reasonability of LUT . Our code is available at URL
After revision:  In addition to the target language sentence translation loss, LUT includes two auxiliary supervising signals to guide the acoustic encoder to extract as much information from the auditory input. In addition, LUT utilizes a pre-trained BERT model to enforce the upper encoder to produce as much semantic information as possible, without extra data. We perform experiments on a diverse set of speech translation benchmarks, including Librispeech English-French, English-German and English-Chinese speech translation benchmarks and the results demonstrate the reasonability of LUT . Our code is available at URL