INTERSPEECH 2021
30 August – 3 September, 2021, Brno, Czechia

ASR Posterior-based Loss for Multi-task End-to-end Speech Translation
Yuka Ko1 , Katsuhito Sudoh1,2 , Sakriani Sakti1,2 , Satoshi Nakamura1,2
1

2

Nara Institute of Science and Technology, JapanRIKEN Center for Advanced Intelligence Project (AIP), Japan{ko.yuka.kp2, sudoh, ssakti, s-nakamura}@is.naist.jp

Abstract

while the standard CE loss only focuses on single referencetranscripts.
This work is motivated by the work of Osamura et al. [6],which proposed robust cascade ST using ASR word posterior
distributions as input for considering ASR output confusion.
Our work extends this into the recent multi-task end-to-end ST
framework. From another perspective, Chuang et al. [7] employed ASR loss using cosine similarity to consider semantic
similarity. The use of distributional loss function in the proposed method is motivated by that work, but our work focuseson the ASR confusion due to pronunciation similarity.
Our proposed method can also be regarded as knowledge
distillation [8] using a pre-trained ASR model. Liu et al. [9]employed knowledge distillation in ST using a pre-trained MTmodel as a teacher. Gaido et al. [10] employed knowledge distillation in ST using a pre-trained MT model and ASR modelwith a loss function based on connectionist temporal classification (CTC) [11] in multi-task learning. Our work does not relyon an additional dataset and focuses on the ASR subtask using
ASR posterior distributions to include possible ASR confusions
into the ST training.
Experimental results in Fisher Spanish-to-English showthat the proposed method resulted in better BLEU scores thanthe baseline with the standard CE loss with label smoothing.

End-to-end speech translation (ST) translates source language
speech directly into target language without an intermediate
automatic speech recognition (ASR) output, as in a cascading
approach. End-to-end ST has the advantage of avoiding error
propagation from the intermediate ASR results, but its performance still lags behind the cascading approach. A recent effort
to increase performance is multi-task learning using an auxiliary task of ASR. However, previous multi-task learning forend-to-end ST using cross entropy (CE) loss in ASR-task targets one-hot references and does not consider ASR confusion.
In this study, we propose a novel end-to-end ST training method
using ASR loss against ASR posterior distributions given bya pre-trained model, which we call ASR posterior-based loss.
The proposed method is expected to consider possible ASR
confusion due to competing hypotheses with similar pronunciations. The proposed method demonstrated better BLEU resultsin our Fisher Spanish-to-English translation experiments thanthe baseline with standard CE loss with label smoothing.
Index Terms: end-to-end speech translation, multi-task learning, spoken language translation

1. Introduction
Speech translation (ST) systems translate source language
speech to target language text. A simple approach is to cascade automatic speech recognition (ASR) and machine translation (MT), but this propagates ASR errors to MT. It is crucial
to develop a robust MT system against ASR errors. A major
approach to tackling this problem is to consider many ASR hypotheses in N-best and lattice form [1].
Recent ST studies attempt end-to-end ST without an explicit ASR module to obtain source language speech transcripts.
The end-to-end approach is promising because it is basically
free from ASR error propagation. However, the translation performance of a simple end-to-end ST trained using source language speech and target language translations is usually worse
than a cascade ST. Multi-task learning [2, 3, 4, 5] is a promising approach to filling the gap between cascade and end-to-end
ST by the use of an additional ASR-based loss function during
training as an ASR subtask. Loss function for the ASR-task isusually implemented using cross entropy (CE) loss against reference transcriptions. However, the previous multi-task learning for end-to-end ST does not consider ASR confusion in ASRtask training. Since our objective is to obtain correct translations, we do not always have to make such hard decisions in an
auxiliary ASR-task.
In this study, we propose a novel training method for endto-end ST using ASR loss against given by a pre-trained ASR
model as reference for prediction in an ASR subtask. We callthis loss function ASR posterior-based loss. We can include
possible ASR confusion using the ASR posterior-based loss,

Copyright © 2021 ISCA

2. End-to-end Speech Translation
2.1. Single-task End-to-end Speech Translation
An end-to-end ST model consists of a source language speechencoder and a target language text decoder. Let X =(x1 , ..., xT ) be a source speech feature sequence and Y =(y1 , ..., yN ) be a target language sequence. Here, T is thelength of the speech frame and N is the length of the target
text, usually in the number of characters or subwords. For eachelement v in the target language vocabulary V , the posterior
probability of v at the i-th symbol in Y can be denoted as:
PST (yi = v) = p(v|X, y<i ).
Its loss function is defined using cross entropy as:
LST = −N X
VX

δ(v, yi ) log PST (yi = v),

(1)

(2)

i=1 v∈V

where δ(v, yi ) is an indicator function that returns 1 if v = yi
and otherwise 0. Recent studies usually apply label smoothing
to avoid overfit, which distributes the probability mass onto theother elements in V .
2.2. Multi-task End-to-end Speech Translation
Single-task end-to-end ST does not have an explicit ASR module and cannot include any teacher signals for ASR. The multitask approach introduces another decoder for ASR using the2272

http://dx.doi.org/10.21437/Interspeech.2021-1105

ASR posterior distribution (soft loss)

One-hot reference (hard loss)

ST-task

ASR-task

ST-taskTargetText

SourceText

TargetText

TargetDecoder

SourceDecoder

TargetDecoder

ଵ

ଶ

்

ଵ

Encoder

ଶ

0

0.02

0.38

0.40

0.01

TargetText
TargetDecoder

ଶ

0.01

0.5

0.4

0.01

0.02

0.38

0.4

0.01

Pre-trained
ASR

SourceDecoderଵ்TargetText
TargetDecoderଶ்

Encoder

Encoder

Figure 2: Hard loss (left) and soft loss (right).
Table 1: WERs by our pre-trained ASR model. WERs reported
in ESPnet document are also shown for reference.

hidden vectors given by the speech encoder. The ASR-task canbe seen as a subtask for the ST problem and its loss function isdefined as:δ(v, yi ) log PASR (yi = v).

0

ଵ

்

Figure 1: Single-task ST (left) and multi-task ST (right).

N X
V
X

1

SourceDecoder

Encoder

LASR = −0

Model
Our Model
ESPnet

(3)

dev
30.2
24.2

dev2
29.1
23.6

test
27.2
22.8

i=1 v∈V

Finally, we define the overall ASR loss as our proposed ASR
posterior-based loss which is a weighted mixture of the hardand soft losses as follows:

The overall loss function is a weighted sum of the two loss functions LST and LASR defined as follows using a weight λASR :L = (1 − λASR )LST + λASR LASR .

(4)
LASR = (1 − λsoft )Lhard + λsoft Lsoft ,We illustrate the single-task ST and multi-task ST in Fig. 1.
In later sections, we refer to the ASR loss LASR as Lhard to differentiate it from the proposed one.

(6)

where λsoft is a weight to control their contributions in the training. Here, λsoft = 0.0 means that the proposed loss function
becomes equivalent to the general CE loss.

3. Proposed Method

4. Experimental Setup

3.1. ASR Posterior-based Loss

We conducted the following experiments to investigate the effectiveness of the ASR posterior-based loss. We used the Fisher
Spanish corpus [12]. This corpus consists of approximately
140K pairs, 160 hours of conversational Spanish speech alongwith its transcriptions and corresponding English translations.
We used it for both ASR and ST model training and developed
Spanish-English (Es-En) speech-to-text translation models.
For data pre-processing in all languages, we lowercasedand normalized punctuation, followed by tokenization with thetokenizer.perl script in the Moses toolkit1 [13]. We used
80-channel log-Mel filterbank coefficients with 3-dimensional
pitch features using Kaldi [14], resulting in 83-dimensional features per frame. The features were normalized by the mean andstandard deviation for each training set. We augmented speech
data by a factor of 3 by speed perturbation [15]. We removed
utterances having more than 3000 frames or 400 characters.
We used subwords for text segmentation for both Spanishand English based on SentencePiece [16], with a shared subword vocabulary with a maximum 1000 entries. The subword
model was trained using the training data and applied to all text
data. The ST and pre-trained ASR models were based on Transformer [17] and implemented using ESPnet2 [18]. We used asingle random seed of 1 in training.
Finally, we applied model averaging with the best fivemodels among 30 training epochs according to BLEU [19]in the Fisher dev (3.9k pairs) set using beam search with

We propose a method to train an ST model in a multi-task manner using a loss function based on posterior distributions given
by a pre-trained ASR model instead of single reference tokens
from the gold standard transcripts. Fig. 2 illustrates the difference between a standard hard loss calculation and the proposed
soft one. Here, the ASR decoder should provide large probabilities to hypotheses with similar pronunciations, in practice.
In contrast, the standard CE loss with single reference tokens,
namely hard loss, does not take such situations into account.
The proposed loss function is defined over ASR posterior
distributions as references to include the ASR confusion into thehidden vectors in the ST model. The proposed training encourages the model to obtain posterior distributions similar to thepre-trained model in the ASR subtask, as in word-level knowledge distillation [8]. As a result, the ST decoder has to handle
the ASR confusion in its training and is expected to be morerobust against possible implicit ASR errors in the end-to-end
ST.
The posterior distributions are given by a pre-trained ASR
model as the outputs from the softmax calculation. Let us define
the posterior probability of an ASR token hypothesis v at the ith position of the ASR result as Psoft (i, v). The Lsoft is defined
as:
N X
V
X
Lsoft = −Psoft (i, v) log PASR (yi = v).
(5)i=1 v∈V

Note that PASR is obtained from the ASR decoder in the STmodel and differs from Psoft from the pre-trained ASR model.

1 https://github.com/moses-smt/mosesdecoder
2 https://github.com/espnet/espnet

2273

Table 2: Results measured in BLEU on Fisher with the hyperparameters λASR and λsoft resulting the best on dev.4

Task
Single-task ST
Multi-task ST

Model
ASR-task loss
CE
CE + Label smoothing
ASR Posterior-based loss (Proposed)

λASR
0.4
0.5
0.4

λsoft
0.5

dev
41.10
44.50
45.29
45.54

BLEU
dev2
41.61
46.20
46.34
46.46

test
40.66
44.88
45.16
45.64

Table 3: Results measured in BLEU on Fisher with different λASR . λsoft were the ones resulting the best on dev.

Task
Single-task ST

Multi-task ST

Model
ASR-task loss
CE
CE + Label smoothing
ASR Posterior-based loss (Proposed)CE
CE + Label smoothing
ASR Posterior-based loss (Proposed)CE
CE + Label smoothing
ASR Posterior-based loss (Proposed)

λASR
0.3

0.4

0.5

λsoft
0.9
0.5
0.5

dev
41.10
44.08
44.23
44.99
44.50
44.61
45.54
43.64
45.29
45.46

BLEU
dev2
41.61
45.07
45.25
45.73
46.20
45.30
46.46
45.28
46.34
46.37

test
40.66
44.69
44.21
44.66
44.88
45.01
45.64
43.83
45.16
46.04

the beam size of 10. The averaged model was applied forFisher dev2 (3.9k pairs) and test (3.6k pairs) data for evaluation. We evaluated on 4-references case-insensitive BLEU withmulti-bleu.detok.perl in Moses.

4.3. Baseline Models

4.1. Pre-trained ASR Model

4.4. Proposed Models5

The pre-trained ASR model was trained using the pairs of Spanish speech and transcripts in the training data. The hyperparameters of the model almost followed the default settings of
ESPnet. The encoder was a 12-layer 2048-dimensional transformer with 6-head 256-dimensional attention with a 6-layer
2048-dimensional decoder. The label smoothing weight for CE
loss was 0.1. Batch size was 64 and accumgrad size was 2.
We chose the best parameter set according to the subword accuracy in the development set from among 30 training epochs.
Table 1 shows the performance in word error rate (WER) of the
pre-trained ASR model using beam search with a beam size of
10. We used this pre-trained ASR model to generate teacher
distributions with greedy search for time efficiency in the decoding of the whole training set. Table 1 also shows the performance mentioned in the ESPnet document3 . We can see that theperformance of our model is worse. The reason would be thedifference in the model configuration; the reported results werefrom the model trained using CTC-based loss.

We chose hyperparameter values according to BLEU in the development set. We chose λASR in the equation 4 from among{0.3, 0.4, 0.5} and chose λsoft in the equation 6 from among{0.1, 0.3, 0.5, 0.7, 0.9, 1.0}.

We compared the proposed method with two baselines: a basic multi-task ST with standard CE loss and that applying labelsmoothing with a weight of 0.1.

5. Results and Discussion
Table 2 shows the performance of the ST models in BLEU
scores with the best hyperparameter values in the dev set. The
results clearly show that the proposed method outperformed thebaselines. Table 3 shows the performance of the ST models
for different λASR . The results also show most of the proposedmodels outperformed the baselines.
For a detailed analysis, Fig. 3 shows the BLEU results on
the Fisher test set on different λsoft in each λASR . The proposed
method achieved the best BLEU scores with λsoft = 0.5 andshowed degradation with the other weights. This was also observed for the Fisher dev2 results, which are excluded from thefigure for simplicity. One possible reason for the degradation is
the performance of the pre-trained ASR model. It was not goodenough to fully depend on the soft loss, so the mixed use of softand hard loss was important. This also suggests that the proposed method works effectively even when the pre-trained ASR
is not good enough. There are differences in the loss values inthe training time. Fig. 4 shows the validation loss values with

4.2. ST Model
The ST model was configured almost the same as the ASR
model above. One major difference was the existence of two
decoders for the ST maintask and the ASR subtask. We appliedlabel smoothing for the ST-task with a weight of 0.1. We set thebatch size to 64 and accumgrad size to 4.

4 Our multi-task ST performance has some gaps between ASR-task
multi-task ST model in ESPnet [20] with CTC-based loss. We focus
on using the attention-based loss calculation method and apply it to our
proposed method.
5 Our code is available at: https://github.com/ahclab/
st-asrpbl

3 https://github.com/espnet/espnet/blob/master/egs/fisher_callhome_spanish/asr1b/
RESULTS.md

2274

ASR posterior-based loss

46.5

46.5

CE

CE + Label smoothing

46.5

46.5

46.04

45.64

45.29

45.5

44.81

44.61
44.5

44.84
44.66

44.5

44.69

44.36
44.57

0.1

0.3

44.5

43.5

43.5

0.5 45.28
0.7
0.9 1.0
45.5

0.1

0.3

0.5

𝜆

0.7

(a) λASR = 0.3

45.00
44.53
44.31

44.90

43.83
0.1

0.9 1.0

0.3

0.5

0.7

0.9 1.0

ୱ୭୤୲

ୱ୭୤୲

BLEU

45.16

44.35

44.21
43.5

45.5 45.28

46.0444.78

45.01 45.13
44.88

BLEU

45.04

BLEU

BLEU

45.5

45.00

(b) λASR = 0.4(c) λASR = 0.5

45.16 Figure 3: Fisher test BLEU on different λsoft in each λASR .

44.53

44.90
44.5

val_loss

120

120

100

100

80

80

60

60

40

40

20

20

43.5

0

120
100

43.83

0
1

5

10

15
20
epoch

25

(a) λsoft = 0.0 (CE)

30

1

0.1
5
10

80
60

15
20
epoch

25

0.3
30

(b) λsoft 40= 0.3

val_loss_asr

val_loss_st

44.31

120

120

120

100

100

100

80

80

80

60

60

60

40

40

40

20

20

20

0
1

5

10

0.5

15
20
epoch

0
25

30

1

(c) λsoft = 0.5

0.7
5
10

15
20
epoch

0.930

25

(d) λsoft = 0.7

0
1

1.0
5

10

15
20
epoch

25

30

(e) λsoft = 1.0

20
Figure4: Validation loss on each λsoft in λASR = 0.4.
0
1

5

10

λASR = 0.4 and varying λsoft . The total validation loss became
larger with larger λsoft . This suggests that the proposed method
introduced larger ASR confusion in the training that might haveworked as regularization. From Fig. 3 and Fig. 4, giving a toolarge weight on the soft loss degraded translation quality, but
the appropriate mixture with the hard loss was effective for ST.
For analysis of output sentences between the baseline andproposed model, Table 4 shows some translation examples
given by the models of the baseline CE loss with label smoothing and proposed ASR posterior-based loss shown in Table 2.
We didn’t have ASR results from ASR-task in this paper. From
Example 1, in the baseline model, the word relationships appears instead of the reference word relaxation. This would be
due to ASR confusion with the Spanish word relaciones. On theother hand, the proposed method predicted the correct word. It
appears that the model is more robust for ASR confusion. From
Example 2, sobrino (Spanish for nephew) would be translated
to nephew in English also. Sobrino is very similar to sobrina
in Spanish. As a result, the ASR-task on the baseline model
predicted the incorrect word sobrino for the correct word sobrina, and the translation was also affected by the error. In contrast, the proposed model gave the correct translation. The proposed method enabled robust ST for ASR confusion using ASR
posterior-based loss.

15
epoch

20

25

30

Table 4: Fisher test examples in Fig. 2 settings.
Example 1
Ground Truth (Es)
Ground Truth (En)
CE + Label smoothing (En)ASR Posterior-based loss (En)para relajaciónfor relaxation
for relationships
for relaxing

Example 2
Ground Truth (Es)
Ground Truth (En)
CE + Label smoothing (En)ASR Posterior-based loss (En)quién no su sobrina
who no your niece
who his nephews
who are your nieces

Our experimental results showed the effectiveness of the proposed method compared with the baselines with standard CE
loss with label smoothing. The results also suggest that mixed
use of the proposed and standard loss is important rather thanusing either one of them.
Future work includes further investigation with different
pre-trained ASR models and other language pairs. The extension to using phonetic information like Salesky et al. [21] inloss calculation for making robust ST against acoustic similar
tokens is also a promising future direction.

6. Conclusions

7. Acknowledgements

In this paper, we proposed ASR posterior-based loss to handle ASR confusion in multi-task end-to-end ST. The proposed
loss function works as knowledge distillation from a pre-trained
ASR model and encourages robust ST against ASR confusion.

Part of this work is supported by JSPS KAKENHI Grant Numbers JP17H06101.

2275

8. References[13] P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico,
N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer,
O. Bojar, A. Constantin, and E. Herbst, “Moses: Open Source
Toolkit for Statistical Machine Translation,” in ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic, J. A. Carroll, A. van den Bosch, and A. Zaenen, Eds. The
Association for Computational Linguistics, 2007.

[1] M. Sperber, G. Neubig, N. Pham, and A. Waibel, “SelfAttentional Models for Lattice Inputs,” in Proceedings of the57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume
1: Long Papers, A. Korhonen, D. R. Traum, and L. Màrquez, Eds.
Association for Computational Linguistics, 2019, pp. 1185–1197.
[2] A. Anastasopoulos and D. Chiang, “Tied Multitask Learning for
Neural Speech Translation,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACLHLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume
1 (Long Papers), M. A. Walker, H. Ji, and A. Stent, Eds. Association for Computational Linguistics, 2018, pp. 82–91.

[14] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek,
N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz et al.,“The Kaldi speech recognition toolkit,” in IEEE 2011 workshop
on automatic speech recognition and understanding, no. CONF.
IEEE Signal Processing Society, 2011.
[15] T. Ko, V. Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition,” in INTERSPEECH 2015, 16th Annual Conference of the International Speech Communication Association, Dresden, Germany, September 6-10, 2015. ISCA,
2015, pp. 3586–3589.

[3] Y. Jia, R. J. Weiss, F. Biadsy, W. Macherey, M. Johnson,
Z. Chen, and Y. Wu, “Direct Speech-to-Speech Translation witha Sequence-to-Sequence Model,” in Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria, 15-19 September 2019, G. Kubin and
Z. Kacic, Eds. ISCA, 2019, pp. 1123–1127.

[16] T. Kudo and J. Richardson, “SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural
Text Processing,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2018, pp. 66–71.

[4] T. Kano, S. Sakti, and S. Nakamura, “End-to-End Speech Translation With Transcoding by Multi-Task Learning for Distant Language Pairs,” IEEE ACM Trans. Audio Speech Lang. Process.,
vol. 28, pp. 1342–1355, 2020.

[17] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N.
Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All You Need,”
in Advances in neural information processing systems, 2017, pp.
5998–6008.

[5] T. Kano, S. Sakti, and S. Nakamura, “Transformer-Based Direct
Speech-To-Speech Translation with Transcoder,” in 2021 IEEE
Spoken Language Technology Workshop (SLT), 2021, pp. 958–
965.

[18] S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno,
N.-E. Y. Soplin, J. Heymann, M. Wiesner, N. Chen et al., “ESPnet: End-to-End Speech Processing Toolkit,” Proc. Interspeech
2018, pp. 2207–2211, 2018.

[6] K. Osamura, T. Kano, S. Sakti, K. Sudoh, and S. Nakamura, “Using Spoken Word Posterior Features in Neural Machine Translation,” Proceedings of the 15th International Workshop on Spoken
Language Translation, 181-188, Oct. 2018, vol. 21, p. 22, 2018.

[19] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: amethod for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318.

[7] S. Chuang, T. Sung, A. H. Liu, and H. Lee, “Worse WER, but
Better BLEU? Leveraging Word Embedding as Intermediate inMultitask End-to-End Speech Translation,” in Proceedings of the58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, D. Jurafsky, J. Chai,
N. Schluter, and J. R. Tetreault, Eds. Association for Computational Linguistics, 2020, pp. 5998–6003.

[20] H. Inaguma, S. Kiyono, K. Duh, S. Karita, N. Yalta, T. Hayashi,and S. Watanabe, “ESPnet-ST: All-in-One Speech Translation
Toolkit,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations,
ACL 2020, Online, July 5-10, 2020, A. Celikyilmaz and T. Wen,
Eds. Association for Computational Linguistics, 2020, pp. 302–
311.

[8] G. E. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge
in a Neural Network,” CoRR, vol. abs/1503.02531, 2015.
[Online]. Available: http://arxiv.org/abs/1503.02531[21] E. Salesky and A. W. Black, “Phone Features Improve Speech
Translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July
5-10, 2020, D. Jurafsky, J. Chai, N. Schluter, and J. R. Tetreault,
Eds. Association for Computational Linguistics, 2020, pp. 2388–
2397.

[9] Y. Liu, H. Xiong, J. Zhang, Z. He, H. Wu, H. Wang, and C. Zong,“End-to-End Speech Translation with Knowledge Distillation,”in Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria, 15-19
September 2019, G. Kubin and Z. Kacic, Eds. ISCA, 2019, pp.
1128–1132.
[10] M. Gaido, M. A. D. Gangi, M. Negri, and M. Turchi,“End-to-End Speech-Translation with Knowledge Distillation:
FBK@IWSLT2020,” in Proceedings of the 17th International
Conference on Spoken Language Translation, IWSLT 2020, Online, July 9 - 10, 2020, M. Federico, A. Waibel, K. Knight,
S. Nakamura, H. Ney, J. Niehues, S. Stüker, D. Wu, J. Mariani,and F. Yvon, Eds. Association for Computational Linguistics,
2020, pp. 80–88.
[11] A. Graves, S. Fernández, F. J. Gomez, and J. Schmidhuber,“Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Machine Learning, Proceedings of the Twenty-Third International Conference(ICML 2006), Pittsburgh, Pennsylvania, USA, June 25-29, 2006,
ser. ACM International Conference Proceeding Series, W. W. Cohen and A. W. Moore, Eds., vol. 148. ACM, 2006, pp. 369–376.
[12] C. Cieri, D. Miller, and K. Walker, “The Fisher Corpus: a Resource for the Next Generations of Speech-to-Text,” in Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon,
Portugal. European Language Resources Association, 2004.

2276