Revisiting the Entropy Semiring for Neural Speech Recognition
Authors: Oscar Chang
Dongseong Hwang
Olivier Siohan
Venue: ICLR 2023 poster (2023)
Keywords: semiring
asr
ctc
rnn-t
entropy
regularization
distillation
streaming
speech recognition

Abstract: In streaming settings, speech recognition models have to map sub-sequences of speech to text before the full audio stream becomes available. However, since alignment information between speech and text is rarely available during training, models need to learn it in a completely self-supervised way. In practice, the exponential number of possible alignments makes this extremely challenging, with models often learning peaky or sub-optimal alignments. Prima facie, the exponential nature of the alignment space makes it difficult to even quantify the uncertainty of a model's alignment distribution. Fortunately, it has been known for decades that the entropy of a probabilistic finite state transducer can be computed in time linear to the size of the transducer via a dynamic programming reduction based on semirings. In this work, we revisit the entropy semiring for neural speech recognition models, and show how alignment entropy can be used to supervise models through regularization or distillation. We also contribute an open-source implementation of CTC and RNN-T in the semiring framework that includes numerically stable and highly parallel variants of the entropy semiring. Empirically, we observe that the addition of alignment distillation improves the accuracy and latency of an already well-optimized teacher-student distillation model, achieving state-of-the-art performance on the Librispeech dataset in the streaming scenario.

Published as a conference paper at ICLR 2023
REVISITING THE ENTROPY SEMIRING FOR NEURAL SPEECH RECOGNITION
Oscar Chang, Dongseong Hwang, Olivier Siohan
Google
(oscarchang, dongseong, s sichan (eoaaannngoogee.oo
ABSTRACT
In streaming settings, speech recognition models have to map sub-sequences of speech to text before the full audio stream becomes available. However, since alignment information between speech and text is rarely available during training. models need learn it in a completely self-suue way. practice, the ex- ponential number of possible alignments makes this extremely challenging, with models often learning peaky or sub-optimal alignments. Prima facie, the exponen- tial nature of the alignment space makes it difficult to even quantify the uncertainty model's aligmment distribution. Fortunately, it has been known for decades that the entropy of probabilistic finite state transducer can be computed in time linear to the size of the transducer via a dynamic programming reduction based semirings. In this work, revisit the entropy semiring for neural speech recognition models, and show how alignment entropy can be used to supervise models through regularization distillation. We also contribute implementation CTC and RNN-T in the semiring framework that includes nu- merically stable and highly parallel variants of the entropy semiring. Empirically, we observe that the addition of alignment distillation improves the accuracy and latency of an already well-optimized teacher- stuuent distillation model, achieving state-of- tatte-ofthh-aaed
open-source
ttt-hh-ar art performance on the Librispeech dataset in the streaming scenario.
INTRODUCTION
Modern automatic speech recognition (ASR) systems deploy single neural network trained in an end-to-end differentiable manner paired corpus of speech and text (Graves et al. 2006 Graves 2012 Chan et al.) 2015) Sak et al. 2017 He et al. 2019)). For many applications like providing closed captions in online meetings or understanding natural language queries for smart assistants, is imperative that an ASR model operates in a streaming fashion with low latency. This means that before the full audio stream becomes available, the model has produce partial recognition outputs that correspond to the already given speech.
Ground truth alignments that annotate sub-sequences of speech with sub-sequences of text are hard to collect, and rarely available in sufficient quantities to be used as training data. Thus, ASR models have to learn alignments from paired examples of un-annotated speech and text in completely self-supervised way. The two most popular alignment models used for neural speech recognition today are Connectionist Temporal Classification (CTC) (Graves et al., 2006) and Recurrent Neural Network Transducer (RNN-T) (Graves, (2022, He et al. (2019). They formulate probabilistic model over the alignment space, and are trained with a negative log-likelihood (NLL) criterion.
Despite the widespread use of CTC and RNN-T, ASR models tend to converge to peaky sub- optimal alignment distributions practice (Miao et al. 20155 Liu et al. 2018: Yu et al.) (02T). Prior work outside of ASR has discovered that the standard NLL loss generally leads over-confident predictions (Pereyra et al.) 20177] Xu et al., (020). Even within ASR, there is strong theoretical evidence that sub-optimal alignments Zeyer et al. 2021 Blondel et al. 2021).
inevitable consequence of the NLL training criterion
A common remedy for mitigating over-confident predictions is to impose entropy regularizer to encourage diversification. For example, label smoothing leads to more calibrated representations and better predictions (Müller et al. 2019 Meister al. 2020). Another popular technique is

---

Published as a conference paper at ICLR 2023
A
01
C
C8
41
02
04
33
0.
Consume
Figure 1: Example of loop-skewed RNN-T lattice annotated with transition probabilities. In this example, there are 3 text tokens to be produced and 4 acoustic tokens to be consumed, which results 31 transitions. The naive calculation of entropy requires 35 * (3 + + 1) = 280 multiplications and 35 - = 34 additions, while the
in total of (3+4) 35 alignments and 2 * 4 +
3tg!
entropy semiring calculation requires 31 93 multiplications and 2 * 4 31 55 additions.
knowledge distillation, where minimize the relative entropy between teacher's and student's soft predictions, instead training on hard labels (Hinton et al.] 2015: Stanton et al.) (02I). How- ever, it is not straightforward to calculate the entropy of the alignment distribution of neural speech model. This because the total number possible alignments is exponential in the length of the acoustic and text sequence, which makes naive calculation intractable.
Fortunately, classical results from Eisner (001); Cortes et al. (2066) show that the entropy of probabilistic finite state transducer can computed in time linear the size of the transducer. Their approach is based on the semiring framework for dynamic programming (Mohri 1998). which generalizes classical algorithms like forward- barward-backward. backward, inside-outside, Viterbi, and belief propaga- tion. The unifying algebraic structure of all these classical algorithms is that state transitions and merges can be interpreted as generalized multiplication and addition operations on semiring. [is- ner (200T); Cortes et al. (2066) ingeniously constructed semiring that corresponds to the compu- tation of entropy.
While these results have been long established, open-source implementations like the OpenIST library (Allauzen et al. 2007) have not been designed with modern automatic differentiation and deep learning libraries in mind. This is why thus far, supervising training with alignment entropy regularization or distillation has not been part of the standard neural speech recognition toolbox. In fact, implementing the entropy semiring on top of modern ASR lattices like CTC or RNN-T is highly non-trivial. During training, we need to compute not only the entropy in the forward pass, but also its gradients in the backward pass, which necessitates highly numerically stable implementation. Note that even for simple operations like calculating the binary cross entropy of a softmax distribution, naive implementations that do not use the LogSumExp trick are plagued by numerical inaccuracies. When it comes to calculating the entropy of a sequence that might be thousands of tokens long, such inaccuracies accumulate quickly, leading to NaNs that make training impossible. Moreover, is crucial that the addition of the alignment entropy supervision does not incur additional forward passes through the ASR lattice beyond the one already done to compute NLL.
Our Contributions We contribute an open-source implementation of CTC and RNN-T in the semiring framework that is both numerically stable and highly parallel. Regarding numerical sta- bility, we find that the vanilla entropy semiring from Eisner (2001); Cortes et (2006) produces unstable outputs not just at the final step, but also during intermediate steps of the dynamic program. Thus, a naive implementation will result instability both the forward and backward pass, since automatic differentiation re-uses activations produced during the intermediate steps. To address this, we propose a novel variant of the entropy semiring that is isomorphic to the original, but is designed to be numerically stable for both the forward and backward pass. Regarding parallelism, our im- plementation allows for efficient plug-and aad--add-yayy computations of arbitrary semirings in the dynamic programming graphs of CTC and RNN-T. Thus, when outputs from more than semiring desired, we can compute them in parallel using a single pass over the same data, by simply plugging in semiring that is formed via the concatenation of existing semirings.
2

---

Published as a conference paper at ICLR 2023
Beyond our open-source contribution, also experimentally validate the effectiveness of align- ment supervision for regularization and distillation in Section Our first experiment targets small- capacity models that are more likely to learn sub-optimal alignments. In Section [.1] we show how alignment entropy regularization can reduce the word error rates (WER) of small LSTTM models by up to 6.5% and small Conformer models by up 2.9% relative the baseline. These results high- light that the optimization objective can significantly impact the performance of the same model. Next in Section 5.2 we propose novel distillation objective, which we term semiring distillation. Under this objective, the teacher model uses the uncertainty of both the token predictions and the alignments to supervise the student. When applied to well-optimized 0.6B parameter RNN-T model, the combined uncertainty produces better WER results in the student than either uncertainty term alone, achieving state-of-the-ard results on the Librispeech dataset in the streaming setting. An ablation study further reveals that while distillation in the token space has small negative effect on latency, distillation in the alignment space reduces emission latency significantly.
The rest of the paper is organized as such: Section 2 preeenns the semiring framework for dynamic programming. Section S provides a brief summary of alignment models for neural speech recog- nition. Section 4 discusses applications for regularization and distillation. Finally, sions about related work, proofs, and further analysis can be found in the Appendix.
implementation of the semiring framework. Section 3 showcases
the paper. Extra discus-
SEMIRING FRAMEWORK FOR DYNAMIC PROGRAMMING
We begin by introducing definitions for a semiring, and adapting Mohri (1998)'s 's semiring frame- work for dynamic programming to weighted directed acyclic graphs (DAG). Unlike Mohri (1998), which applies to single-source directed graphs, we consider DAGs with multiple roots, since they appear more commonly in ASR, for example the CTC lattice.
Definition 2.1. A monoid is set M equipped with binary associative operation c i.. Va, b, E M, (a @ b) + c = a o (b O e), and an identity element e E M, i.e. Va E M, 1 a = a. M is called commutative monoid if @ is commutative, i.e. Va, b E M, O boa
Definition 2.2. A semiring is a set I equipped with addition @ and multiplication @ such that:
1. (R, +) is commutative monoid with identity element Ó. 2. (R, @) is monoid with identity element I.
3. & distributes over a from both the left and right, i.e. Va, b R. a @ (b e c) = (a @ b) @ (a & c) and (b @ c) & (b 2 @ (c
4. 0 is an annihilator for (R, 8)) i.e. Va E R. s 0 0 x ū.
Definition 2.3. Let G = (V, E)) be DAG where every edge e E E' is assigned a weight w: E - R. A maximal path of G is path starting from a root of G to one of G's leaves. The weight w((r) of path x is the product of weights of edges in the path. A computation of G is the sum of the weights of all the maximal paths in G, i.e.
C(G, W. 0,0
w(e).
#EMasPatts(C)
Even though MaxPaths(G) might be exponentially big, because @ distributes over @ in a semiring, only 1 multiplication operation is needed for each edge, resulting in linear complexity O(VV El).
Theorem 2.4. For a DAG G = GVVV E), if (w((), (,) &, 0, 1) is a semiring, then C(G, w, e 3 can + (E]) via dynamic programming.
be computed in linear time O(IV
Proof. Sort the vertices in topological order, and enumerate them as v1,..., Tu- Then in that order, inductively accumulate the weights from the edges into the vertices as follows. We slightly abuse notation by over-loading w to also be function that assigns weights to vertices.
Ug is a root. otherwise.
w(x:)
(2)
① (V.,))E w(v) & w(e)
3

---

Published as a conference paper at ICLR 2023
Let II denote the set of paths ending at vertex that start start,from root. We now prove by induction that w(e+) = *eu. W(T). The base case of 01 is true by definition given that it is root. For the inductive step, assume that Vj I k. w(v;) Drell,; w(x)). Observe that
w(ck+1)
w(v) wle))=
-
w(x))
@ w(e)
(=(,,ve+))E
(=(VrEES)EE Arell,
(3)
- ( ((TT (()))
w(x).
(= Axell,
sellett1
The sum of all the weights in the leaves yields the desired computation in single pass over G.
Below, we walk the reader through simple examples to demonstrate how Theorem 2.4 works.
Definition 2.5. The probability semiring can be defined as such: wo(e) = p(e) where p: E - [0, the probability function, +. Example 2.6. Consider the following DAG with the probability semiring.
x, 0 0, I
V1
E1
03
U/ C2
64
Ub
w((1) = m((2)) = 1.
w(va) (wr(v)) & w(e))) e (w()))(((()))))))))))))))))))))) p(en) p(e2)-
w(v4) = w(va) @ w(ea) p(e))p(e))
w(vs) wva)) @ w(e4) p(e))ppe)))
Cp(n) = w((v) @ w(vs) = plpapp(epa)+++))1ppp)))) p(e))e))+ +
+ p(ez)p(es).
In practice, it is common to perform multiplication of probabilities in the log space instead for numerical stability. Under the semiring framework, we can see that this the same dynamic pro- gramming computation but with different semiring.
Definition 2.7. The log semiring can be defined as such: w<e) p(e), a @ log(e" c²),
@ b = + b, 0 = -00 1 = 0.
Example 2.8. Consider the DAG in Example 2.6 with the log semiring. (Details in Appendix C.1
Clgg(()) log (p(er)((es)) p(e2)p(es) p(er)p(es) + p(e2)p(es))-
2.1 ENTROPY SEMIRING
Notice that setting w:(e) = p(e) log with either the probability semiring or the log semiring will not compute the entropy. The salient insight made by Eisner (2001) Cortes et al. (2006) is that entropy can be calculated via semiring that derives its algebraic structure from the dual number system. Dual numbers are hyper-complex numbers that can be expressed as + be such that R, b R, where addition and multiplication are defined as follows.
Addition:
(a+ bc) (c + de) (a c) + (b ddc. + (ad + bc)c.
(4)
Multiplication:
(a +a+ be) x (c+ de) = ac
Observe that dual numbers form a commutative semiring with the additive identity 0 + Oe and multiplicative identity 1 + Oc. Now if we use dual-valued weights w: E + R x IR with the real component representing the probability p(e) and the imaginary component representing the negative entropy p(c) log p(e), the entropy of weighted DAG can be computed as before via Theorem 2.4 Definition 2.9. The entropy semiring can be defined as such:
w(e) = (p(e), log) p(e)),
(a, b) (c, d) (a d),
(a, b) @ (c, d) = (ac, ad + be),
o (0,0), i (1,0).
4

---

Published as a conference paper at ICLR 2023
Example 2.10. Consider the DAG in Example 2.6 pendix C.2)
with the entropy semiring. (Details in Ap-
(or (Ca(a)s (p(e))(es)oog log log p(ea)pes) log ((ee))((es))-
3 NEURAL SPEECH RECOGNITION
Speech recognition is the task of transcribing speech into text y, and can be formulated prob- abilistically as a discriminative model ply(z). Because I and are represented as sequences of vectors, we can define an alignment model that maps sub-sequences alignments allow a speech recognition model to emit partial sub- of given partial sub- sequence z; and thereby work in streaming fashion. Below, we briefly recap CTC and RNN-T, which are the two most widely used alignment models for neural speech recognition.
to sub-sequences of y. The
Let the acoustic input be denoted as x I1.T where It E IRd and the text labels be denoted y y1.0 where Yu E V are vocabulary tokens (we use denote an inclusive range). The likelihood of CTC model is defined as:
T
P(yl=)= = E
II P(be)1))))
(5)
(x.g)) (=1
where y D1:T Acre C [V corresponds to alignments such that removing blanks and repeated symbols from D results in y. CTC makes an assumption of conditional independence, so the likelihoods of every token are independent of each other given the acoustic input.
The likelihood of an RNN-T model is defined as:
T+U Σ II
P(ul=)=
(6)
(E s=1
where U = Uu.T4 E ARNN-T C (V corresponds to alignments such that removing blanks from results in y. Unlike CTC, the likelihood of each token in RNN-T depends on the history of tokens that come before it.
CTC and RNN-T lattices DAGS where the model likelihoods can be computed efficiently via dynamic programming by transforming the sum-prodie form of Eqs. (5) and (6) into product-sum form via Theorem 2.4 Thus, even though the number of alignments is exponential in the length of the acoustic and text input O[1x) (w]), the likelihood and its gradient can be computed in time linear the size of the lattice (1 (g))) In practice, most implementations of CTC and RNN-T compute likelihoods in the log space for numerical stability. which corresponds to dynamic programming using log semiring (cf. Definition 2.7 More broadly, Eqs. (5) and (6) can be recognized as
specific instances of the more general Eq.
P(y(z)
Σ Pm
TEA(z.9)
While we focus on CTC and RNN-T our paper, other examples of neural speech recognition lattices that use dynamic programming to efficiently marginalize over an exponential number of alignments include Auto Segmentation Criterion (Collobert et al. 2016), Lattice-Free MMI (Povey et al. 2016), and the Recurrent Neural Aligner (Sak et al. 2017).
4 IMPLEMENTATION OF THE SEMIRING FRAMEWORK
While the mathematical details of the entropy semiring have been known for more than two decades, it is highly non-trivial to implement them in neural speech recognition models. This is in part because modern deep learning models operate on much longer sequences and have to compute gradients, and in part because of intricate implementation details like applying loop skewing on the RNN-T lattice (Bagby et al. 2018). One of the main contributions our work is to make an open- source implementation of CTC and RNN-T in the semiring framework available to the research community (cf. Supplementary Material). Below, we introduce two variants of the entropy semiring that are designed for numerical stability and parallelism.
5

---

Published as a conference paper at ICLR 2023
4.1 NUMERICAL STABILITY
A numerically stable implementation of the entropy semiring has to avoid the naive multiplication of two small numbers, and do it in log space instead. As such, the mathematical formulation pro- vided in Definition 2.9 is not actually numerically stable because involves multiplying P1 and P2 log P2 for two small numbers P1 P2. Notice that applying log morphism on just the first argu- ment of the dual number is not sufficient to ensure numerical stability in the backward pass because limp-0 apP pogp = limp- + logp = -00.
We take care of numerical stability in both the forward and backward passes by applying log morphism on both arguments of the dual number in the semiring. For the second argument, the log morphism has to be applied on the entropy -p -plog since we cannot take the logarithm of negative numbers. This results in the following variant of the entropy semiring.
Definition 4.1. The log entropy semiring can be defined as such:
w(e) (log log( p(e) lo(()-e)) log
(a, b) @ (c, (log(e" + + log(e' + +)))), (a, (c,d) (a e, log(e + eltc)),
= (-o0,- -o). (0, -00)
4.2 ENTROPY SEMIRING FOR DISTILLATION
Minimizing the negative log likelihood of a parameterized model equivalent to minimizing the KI. divergence between the empirical data distribution and the modeled distribution. Knowledge distillation is a technique that proposes to use soft labels from the output of teacher model in the place of hard labels in the ground truth data (Hinton et al. 2015). It is often implemented weighted sum of two different KL divergences: one between the empirical data distribution and the model (hard labels), and one between the teacher distribution and the model (soft labels).
Eassill KL (P [Pempirriall
| Pttudo)) + KL (P (Pecccerrl
P Pstudont).
(8)
We can re-write the distillation loss in Eq, ( as a sum of three different terms.
Caistlll log Psudeat + Ocdistill [Peacher log P Pleacher
P Pieacher log Psudead]
Notice that the first term can be computed with the log semiring, while the second and third term can be computed with the log entropy semiring. But instead of doing three separate forward passes, can compute the distillation loss using single forward pass by concatenating them to form new semiring weighted by four real number values w: E + R4
Definition 4.2. The log reverse-Ki KL semiring can be defined as such:
wee)= = (logp(e) log g(e), (og(ll(((())))))) log (())), log(-q(e) logg(e()))
(a, b, c, d) @ (f.g. h i) (log(e e log(e" e²), log(e + e²), log(ed d et)), ba,b,c, @ (f.g. h, = (a+ b log(e + eoto), log(ehti + edto)), (-00, 0o, 00), (0,0, 00, -0).
APPLICATIONS
Below, we study two applications of the entropy semiring for neural speech recognition.
5.1 ENTROPY REGULARIZATION
Motivation Because there is an exponential number of alignments, it is difficult in general for an optimization algorithm to settle on the optimal alignment given no explicit supervision. Instead, set of feasible alignments has been found during training, it tends to dominate, and error signals concentrate around the vicinity of such alignments. It has been experimentally observed that both CTC and RNN-T tend to produce highly peaky and over-confident distributions, and converge

---

Published as a conference paper at ICLR 2023
Table 1: Word Error Rate with and without Entropy Regularization.
Model
fPaarams
Method
Dev-Clean
Dev-Other
Test-Clean
Test-Other
CTC LSTM
22M
Baseline
7.7 0.3
21.9 + 0.7
T.7 + 0.3 7.2 + 0.3
21.8 + 0.6 20.8 + 0.6
CTC LSTM
22M
Ent
7.3 + 0.3
20.7 + 0.7
CTC Conformer
9M
Baseline
3.9 0.2 3.9 0.2
10.2 + 0.4
4.1 0.2
10.2 + 0.4
CTC Conformer
Ent
0.9
4.1 111:0 0.2
9.9 0.4
RNN-T LSTM
25M
Baseline Ent
0.8
23.6 0.8
7.4 0.3
24.0 0.8
RNN-T LSTM
25M
7.4
2.8
0.3
23.1 + 0.7
RNN-T Conformer
10M
Baseline Ent
2.5+0.2
0.3
0.2
6.7 6.3
RNN-T Conformer
10M
2.5 0.2
6.3
0.7
towards local optima (Miao et al. 2013 Liu et al. 2018: Yu et al. 2021). There is also theoretical extent inevitable, and direct result of the
analysis that suggests that this phenomenon to training criterion (Zeyer et al. [021]] Blondel et al.) 2021). We propose to ameliorate this prob- lem via entropy regularization mechanism that penalizes the negative entropy of the alignment distribution, which encourages the exploration of more alignment paths during training.
EEt
Σ
log (T) + (ooetmoode log ( (T).
(10)
xEA((..))
Experimental Setup We experimented with models using non-causal LSTM and Conformer (Gu- lati et al., 2020) encoders on the Librispeech dataset (Panayotov et al., 2015). The acoustic input is processed as 50-dimensional log-mel filter bank coefficients using short-time Fourier transform windows size 25ms and stride 10ms. All models are trained with Adam using the optimization schedule specified in Vaswani et al. (2017), with 10k warmup, batch size 2048, and peak learning rate 0.002. The LSTM encoders have bi-directional layers with cell size 512 and are trained for 100k steps, while the Conformer encoders have 16 full-context attention layers with model dimen- sion 144 and are trained for 400k steps. Decoding for all models is done with beam search, with the CTC decoders using beam width of 16, and the RNN-T decoders using beam width of 8 and a 1-layer LSTM with cell size 320. All models use a WordPiece tokenizer with a vocabulary size of 1024. O'ea was selected via a grid search {0.01, 0.001}. We report 95% confidence interval estimates for the WER following Vilar 20088)
Results We from T that adding entropy regularization improves WER performance the baseline model in almost all cases. On LSTMs, the improvements were the biggest, for example there was big WER reduction of 6.5% in the Test-Clean case for the CTC LSTM from 7.7% to 7.2%. The improvements were smaller for the Conformers, for example with the WER remaining the same on Dev-Clean and dropping by an absolute 0.1% on Test-Clean for the RNN-T model. In general, we find that the improvements are the biggest when the baseline performance is the worst, indicating that entropy regularization the most effective when the baseline model has converged sub-optimally and has under-utilized its model capacity. We provide visualizations and further qual- itative analysis in Appendix
5.2 RNN-T DISTILLATION
Motivation Distillation is a technique that a pre-trained teacher model to aid in the training of student model. It can be especially helpful in the supi-isuere setting where a teacher model is trained on small set of labeled data and then used to generate pseudo-labels for larger set of unlabeled data (Park et al. 2020). A student model is then trained using the standard NL.. loss on the combined of labeled and pseudo-labeled data. We refer to this process as hard distillation.
Lrand log Pauuent (ū !teacher!z:)) (z:)
(11)
In addition to the hard pseudo-labels, we can use the soft logits from the teacher model to provide a
better training signal to the student model. Prior work for RNN-T models has implemented this by

---

Published as a conference paper at ICLR 2023
summing up the KL divergence between the state- wise posteriors of both models (Kurata & Saon 2020 Panchapagesan et al. 2021). We refer to this process as soft distillation.
Lson =
Liard Ostane
KLytate =
Σ 1,
Puecche (D++u T1:t V1:u-1) log Pstudemt (Dt+a = L0 211t V1:u-1)
P Pleacher (Dttu Lo F1 V1s--1)
(12)
Streaming models have to carefully balance the competing objectives of reducing emission latency and improving WER accuracy. Alignments that delay emission can leverage more information to perform its prediction, but the cost of latency. Because both the hard and soft distillation objec- tives operate in token space, they neglect potentially helpful alignment information from the teacher model. Prior work has found that alignment information from a good pre-trained model can help guide attention siteention-based encoder-decoder models to achieve both good latency and accuracy (Inaguma & Kawahara 202T). We propose to add a KI. divergence loss for the sequence-wise posteriors of both models to incorporate alignment information into the distillation objective. Because this technique is based on the use of a semiring (cf. Section 4.2). we refer to as semiring distillation.
Lycmiring = Lgostt + aseqg KLoqq
Klaeg
Σ
P Preacher (T) log Pwacher (TT
(13)
Pstadent
xEAANNer(a..))
Experimental Setup We set up a semi-supervised learning scenario by using Libri-Light as the unlabeled dataset (Kahn et al.) 2020) and Librispeech as the labeled dataset. First, we prepare pseudo-labels for Ligri-Light from Zhang et al. (2020b), where an iterative hard distillation process is performed with non-causal 1.0B Conformer model to arrive at high quality pseudo-labels. Then, we train a teacher model by randomly sampling batches from Libri-Light and Librispeech in a 90:10 mix. This teacher model is used our hard distillation baseline, and also used to train student model via soft and semiring distillation for comparison. Both teacher and student models use the same 0.6B causal Conformer architecture in Chiu al. (2022) with self-supervised pre-training on Libri-Light via a random projection quantizer. Like Hwang et al. (2022), the student model has FreqAug applied
for data augmentation. Both models were trained for 160k steps with Adam using the same optimization schedule as in Vaswani et al. (2017), but with peak learning rate of 0.0015, a 5k warmup, and batch size 256. The RNN-T decoder uses a 2-layer LSTM with cell size 1200, beam width 8, and WordPiece tokenizer with vocabulary size of 1024. We do grid search for both O'state and Okeg over (0.01,0.001) We report 95% confidence interval estimates for the WER following Vilar (2008).
Results We report our results in Table 2 We see that soft distillation results in substantial WER improvements over the hard distillation baseline, reducing WER on the Test-Clean set by 22% from 2.7% to 2.1% and on the small WER improvements over soft distillation, further reducing WER by an absolute 0.1% on all the Dev and Test sets. We further do an ablation study with semiring distillation using Ostate 0.0, and find that aligmment distillation alone, while under-pertee unnder-perfoming soft distillation, still results in significant WER improvements over the hard distillation baseline. These results suggest that while good measure of uncertainty in the token space helps improve accuracy more than a corresponding measure of uncertainty in the alignment space, these two measures of uncertainty are complementary to each other and should be used jointly for best results.
set by 17% from 6.4% to 3.3%. Semiring distillation results in
We do relative latency measurements between two models following Chiu al. (2022) ). Start and end times of every word in the output hypotheses are first calculated, and then used to compute the average word timing difference between matching words from the hypotheses of both models. Interestingly, we see that even though the soft distillation model obtained better WER accuracy, this came at a slight cost to emission latency (+2.7m 7ms) relative to the baseline hard distillation model. On the other hand, the Osstate = 0.0 semiring distillation ablation performed significantly better in terms of relative latency (-65.7mm 7ms), indicating that alignment information from the teacher helped the student learn to emit tokens faster. By combining both state-wise and sequence-wis wise posteriors, the semiring distillation model improved on both emission latency and WER accuracy compared to the hard or soft distillation model.
8

---

Published as a conference paper at ICLR 2023
Table 2: Self-Distillation for a 0.6B Causal Conformer RNN-T Model.
Distillation Method
Relative Latency (ms)
Dev-Clean
Dev-Other
Test-Clean
Test-Other
Hard Distillation
0.0
2.5 0.2
6.8 + 0.4
2.7 0.2 2.1 0.2
6.4 0.3
Soft Distillation
+2.7
1.9 1.9+0. 0.1
5.3 0.3
5.3 0.3
Semiring Distillation
with Clstate 0.0
65.7
2.0 +
5.5 0.3
2.1 0.2
5.8 + 0.3
with Cistate 0.001
5.2
1.1
5.2 + 0.3
2.0 1: 0.2
5.2 0.3
Table 3: Comparison with Prior Work on Librispeech in the Streaming Setting. Libri-Light: uses unlabeled data from Ligri-Light. Lookahead: the model uses a limited amount future context.
Prior Work
Libri-Light
Lookahead
aParams
Dev-Clean
Dev-Other
Test-Clean
Test-Other
Zhhagg et al.
2020a
No
No
4.2
11.3
Zhang
z020a
No
Yes
3.6
10.0
Yu ct al. (2020)
No
No
30M
3.7
9.2
Cao al. 2021
No
No
3.2
8.5
3.5
8.7
Hwang et al.
2222
Yes
No
0.2B
4.0
9.4
4.4
8.6
Moritz et
(2020)
No
No
2.9
8.1
3.2
8.0
Yu et al. [202T)
No
No
3.1
7.5
Morttz et al.
(2020)
No
Yes
2.7
7.1
2.8
Chuu et al. 2022)
Yes
No
0.6B
2.5
6.9
2.8
6.6
Shi ct al. (0021)
No
Yes
80M
2.4
6.1
Our work
Yes
No
0.6B
1.8
5.2
2.0
5.2
Finally, we compare the performance our semiring-distilled model with prior work for Lib- rispeech in the streaming setting in Table To the best of our knowledge, work has achieved new state-of-ther thte-offthhe-rt on this benchmark, without using any future context in making its predictions.
CONCLUSION
Across multiple machine learning applications in speech and natural language processing, compu- tational biology, and even computer vision, there has been paradigm shift towards sequence-to- sequence modeling using Transformer based architectures. Most learning objectives on these mod- els do not take advantage of the structured nature of the inputs and outputs. This is in stark contrast to the rich literature of sequential modeling approaches based on weighted finite-state transducers from before the deep learning renaissance (Eisner 2002 Mohri et al. 2002 Cortes al. 2004).
Our work draws upon these pre deep learning era approaches to introduce alignment- baase su- pervision for neural ASR in the form of regularization and distillation. We believe that applying similar semiring based techniques to supervise the alignment between data of different modalities or domains will lead to better learning representations. For example, recent work has found that image- text models do not have good visio-linguisti compositional reasoning abilities (Thrush et al., 2022). This problem can potentially be addressed by learning good alignment model between natural language description of an object and its corresponding pixel-level image representation. Another area of future work can be learning to synchronize audio and visual input streams to solve tasks with multiple output streams like overlapping speech recognition diarization.
Finally, we are excited about the gamut of new ASR research that will arise from plugging different semirings into our CTC and RNN-T semiring framework implementation. For example, even though most mainstream approaches use the log semiring for training and decoding, a differentiable relax- ation akin to Cuturi & Blondel (2017) might yield surprising findings. Another promising direction for future work can involve calculating the mean or variance of hypothesis lengths for diagnostic or regularization purposes.

---

Published as a conference paper at ICLR 2023
REFERENCES
Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojeiech Skut, and Mehryar Mohri. Openfst: A general and efficient weighted finite-state transducer library. In International Conference on Implementation and Application of Automata, pp. 11-23. Springer, 2007.
Tom Bagby, Kanishka Rao, and Khe Chai Sim. Efficient implementation of recurrent neural network transducer in tensorflow. In 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 506- 512. IEEE, 2018.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, vol- ume 4. Springer, 2006.
Mathieu Blondel, Arthur Mensch, and Jean-Philippe Vert. Differentiable divergences between time series. In International Conference on Artificial Intelligence and Statistics, pp. 3853-3861.
PMLR, 2021.
Songjun Cao, Yueteng Kang, Yanzhe Fu, Xiaoshuo Xu, Sining Sun, Yike Zhang, and Long Ma. Improving streaming transformer based asr under a framework of self-supervised learning. arXiv preprint arXiv:2109.07327, 2021.
William Chan, Navdeep Jaitly, Quoc V Le, and Oriol Vinyals. Listen, attend and spell. arXiv preprint arXivv1508.00211, 2015.
Chung-Chern Chiu, James Qin, Yu Zhang, Jiahui Yu, and Yonghui Wu. Self-supervised learning with random-projection quantizer for speech recognition. arXiv preprint arXiv:2202.01855, 2022.
Ronan Collobert, Christian Puhrsch, and Gabriel Synnaeve. Wav2letter: end-to-end convnet- based speech recognition system. arXiv preprint /6Xiv:1609.031 193,, 2016.
Corinna Cortes, Patrick Haffner, and Mehryar Mohri. Rational kernels: Theory and algorithms.
Journal of Machine Learning Research, (AAu)):1035-1065 2004.
Corinna Cortes, Mehryar Mohri, Ashish Rastogi. and Michael Riley. Efficient computation of the relative entropy of probabilistic automata. In Proceedings of the 7th Latin American Confer- ence on Theoretical Informatics, LATIN'06, pp. 323-336, Berlin, Heidelberg, 2006. Springer- http:://doi..ggge/
Verlag. ISBN 354032755X.
doi: 10.1007/11688 1682462 32. URL.
I007/11682662_331
Marco Cuturi and Mathicu Blondel. Soft-dtw: a differentiable loss function for time-series. In International conference on machine learning, pp. 894-999--9999 903. PMLR, 2017.
Jason Eisner. Expectation semirings: Flexible em for learning finite-stat state transducers. Proceed- ings of the ESSLLI workshop on finite-state methods in NLP, pp. 1-5, 2001.
Jason Eisner. Parameter estimation for probabilistic finite-state transducers. In Proceedings of the 40th Anmual Meeting of the Association for Computational Linguistics, pp. 1-8, 2002.
Jason Eisner. Inside-outside and forward-backward algorithms are just backprop (tutorial paper). In Proceedings of the Workshop on Structured Prediction for NLP, pp. 1-17, 2016.
António Farinhas, Wilker Aziz, Vlad Niculae, and André FT Martins. Sparse communication via mixed distributions. arXiv preprint arXiv:2108.02633 026588 2021.
Alex Graves. Sequence transduction with recurrent neural networks.
arXiv preprint
12Xiv: 2012.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
Anmol Gulati, James Qin, Chung- Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmentee transformer for speech recognition. arXiv preprint arXiv:2005.088 080000 2020.
10

---

Published as a conference paper at ICLR 2023
Awni Hannun, Vineel Pratap, Jacob Kahn, and Wei-Ning Hsu. Differentiable weighted finite-state transducers. arXiv preprint arXiv:2010.01003 2020.
Yanzhang He, Tara N Sainath, Rohit Prabhavalkar, lan McGraw, Raziel Alvarez, Ding Zhao, David Rybach, Anjuli Kannan, Yonghui Wu, Ruoming Pang, al. Streaming end-to-end speech recog- nition for mobile devices. In ICASSP 2019-2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 6381-6385. IEEE, 2019.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. arXiv preprint urXiv:1503.02531, 2(7), 2015.
Dongseong Hwang. Khe Chai Sim, Yu Zhang, and Trevor Strohman. Comparison of soft and hard target rnn-t distillation for large-scale ast. arXiv preprint urXiv:2210.05793, 2022.
Hirofumi Inaguma and Tatsuya Kawahara. Alignment knowledge distillation for online streaming attention-based speech recognition. arXiv preprint arXiv:2103.00422, 2021. 2021.
Jacob Kahn, Morgane Rivière, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Fierre-Emmanue Mazaré, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, et al. Libri- light: benchmark for asr with limited or no supervision. In ICASSP 2020-2020 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7669-7673, IEEE, 2020.
Gakuto Kurata and George Saon. Knowledge distillation from offline to streaming rnn transducer for end-to-end speech recognition. Interspeech, pp. 2117-2121, 2020.
Zhifei Li and Jason Eisner. First-and second-order expectation semirings with applications to minimum-risk training on translation forests. In Proceedings of the 2009 Conference Em- pirical Methods in Natural Language Processing, pp. 40-51, 2009.
Hu Liu, Sheng Jin, and Changshui Zhang. Connectionist temporal classification with maximum entropy regularization. Advances Neural Information Processing Systems, 31, 2018.
Clara Meister, Elizabeth Salesky, and Ryan Cotterell. Generalized entropy regularization or: There's nothing special about label smoothing. arXiv preprint arXiv:2005.0088 008200 2020.
Arthur Mensch and Mathieu Blondel. Differentiable dynamic programming for structured prediction and attention. In International Conference Machine Learning, pp. 1462-3471. PMLR, 2018.
Yajie Miao, Mohammad Gowayyed, and Florian Metze. Fesen: End-to-end speech recognition using deep models and wist-based decoding. In 2015 IEEE Workshop Automatic Speech Recognition and Understanding (ASRU), pp. 167-174. IEEE, 2015.
Mehryar Mohri. General algebraic frameworks and algorithms for shortest-distance problems. Tech- nical report, Technical Memorandum 981210 10TM, AT&T Rebs-Research, 62 pages, 1998.
Mehryar Mohri, Fernando Pereira, and Michael Riley. Weighted finite-state transducers in speech recognition. Computer Speech & Language, 16(1):66 (6(1):69-88, 2002.
Niko Moritz, Takaaki Hori, and Jonathan Le. Streaming automatic speech recognition with the transformer model. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 6078- IEEE, 2020.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances neural information processing systems, 32, 2019.
Rani Nelken and Stuart Merrill Shieber. Computing the kullback-leibler divergence between prob- abilistic automata using rational kernels. Technical report, Harvard Computer Science Group Technical Report TR-07-06 2006.
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2206-5210. IEFE, 2015.
11

---

Published as a conference paper at ICLR 2023
Sankaran Panchapagesan, Daniel S Park, Chung- Chengg Chiu, Yuan Shangguan, Qiao Liang, and Alexander Gruenstein. Efficient knowledge distillation for an--raasduuerr models. In ICASSP 2021-2021 IEEE International Conference Acousties, Speech and Signal Processing (ICASSP), pp- 5639-5643. IEEE, 2021.
Daniel Park, Yu Zhang, Ye Jia, Wei Han, Chung-Cheng Chiu, Bo Li, Yonghui Wu, and Quoc V Le. Improved noisy student training for automatic speech recognition. arXiv preprint arXiv:2005.09629 2020.
Gabriel Pereyra, George Tucker, Jan Chorowski, Eukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06544 2017.
Daniel Povey, Vijayaditya Peddinti, Daniel Galvez, Pegah Ghahremani, Vimal Manohar, Xingyu Na, Yiming Wang. and Sanjeev Khudanpur. Purely sequence-trained neural networks for ast based on lattice-free mmi. In Interspeech, pp. 2751-2755, 2016.
Hasim Sak, Matt Shannon, Kanishka Rao, and Françoise Beaufays. Recurrent neural aligner: encoder-decoder neural network model for sequence to sequence mapping. In Interspeech, vol- ume 8, pp. 1298-1302, 2017.
Yangyang Shi, Yongqiang Wang, Chunyang Wu, Ching-Feen Yeh, Julian Chan, Frank Zhang, Duc Le. and Mike Seltzer. Emformer: Efficient memory transformer based acoustic model for low latency streaming speech recognition. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6783-6787. IEEE, 2021.
Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew G Wilson. Does knowledge distillation really work? Advances in Neural Information Processing Systems, 34:6906-699 2021.
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic composi- tionality. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recogni- fion, pp. 9238-5248, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- fion processing systems, 30, 2017.
Juan Miguel Vilar. Efficient computation of confidence intervals forword error rates. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5101-5104. IEEE, 2008
Martin Wainwright, Michael I Jordan, al. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning. 1(1-2):1-305, 2008.
Yi Xu, Yuanhong Xu, Qi Qian, Hao Li, and Rong Jin. Towards understanding label smoothing. arXiv preprint htt.::2006.1155... 2020.
Jiahui Yu, Wei Han, Anmol Gulati, Chung-Cheng Chiu, Bo Li, Tara N Sainath, Yonghui Wu, and Ruoming Pang. Dual-mode asr: Unify and improve streaming asr with full-context modeling. arXiv preprint +xXiv:000000030, 2020.
Jiahui Yu, Chung-Chen Chiu, Bo Li, Shuo-yiin Chang, Tara N Sainath, Yanzhang He, Arun Narayanan, Wei Han, Anmol Gulati, Yonghui Wu, et al. Fastemit: Low-latency streaming asr with sequence-level emission regularization. In ICASSP 2021-2021 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pp. 6004-6008. IEEE, 2021.
Albert Zeyer, Ralf Schlüter, and Hermann Ney. Why does ctc result in peaky behavior? arXiv preprint arXiv:2 105. 14849, 2021.
12

---

Published as a conference paper at ICLR 2023
Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, and Shankar Kumar. Transformer transducer: A streamable speech recognition model with transformer en- coders and mn-t loss. In ICASSP 2020-2020 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7829-7833. IEFF, 2020a.
Yu Zhang, James Qin, Daniel Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V Le, and Yonghui Wu. Pushing the limits of semi-supervised learning for automatic speech recognition. arXiv preprint arXiv:2010. 100004,00.00004, 2020b.
13

---

Published as a conference paper at ICLR 2023
A APPENDIX
The Appendix is organized as follows: Appendix A discusses other related work in the litera- ture. Appendix B provides proofs that the semirings defined our paper are actual semirings. C provides extended derivations for Examples and 2.10 Appendix D showcases vi- sualizations and further qualitative analysis on the effects of entropy regularization.
Appendix
A RELATED WORK
Cortes et (2004) showed how weighted transducers can be used form rational kemelss for sup- port vector machines, thereby expanding their use to classification problems with variable-length in- puts. Thereafter, Nelken & Shieber (2006) demonstrated a rational kernel with the entropy semiring. The use of semirings has been well-established in the conditional random fields literature: replacing the max-plus algebra with the sum-product algebra is a way to derive the neward-backwee
algo-
rithm from Viterbi (Bishop & Nasrabadi] 2006; Wainwright al. 2008). Li & Eisner (2009) studied first-order and second-order expectation semirings with applications to machine translation. Eisner (016) discussed how the inside-outside algorithm can be derived simply from backpropagation and automatic differentiation. Liu et al. (2018) proposed a different CTC-specific entropy regularization objective that based similar motivation of avoiding bad alignments. (Cuturi & Blondel (2017) 7); Mensch & Blondel (2018); BBondell et al. (2021) explore various differentiable relaxations for dy- namic programming based losses deep learning. |Hannun et al. (2020) proposed differentiable weighted finite state transducers, which focused on plug-and-play graphs that support the log and tropical semirings, as opposed to our work, which focuses on plug-and-play semirings that support the CTC and RNN-T graphs. Because our graphs are fixed and coded in Tensorflow itself, we are able to perform graph optimizations (for example, loop-skewing on the RNN-T lattice), compiler optimizations (for example, the XLA compiler for Tensorflow produces production-qualic binaries variety of software and hardware architectures), and even semiring optimizations (for example, proposed variants of the entropy semiring that more numerically stable and highly parallel), thereby ensuring an especially efficient implementation for neural speech recognition. The use of both state- wise and sequence-wise posteriors in semiring distillation is similar to the direct sum en- tropy introduced in Farinhas et al. (202T), which is a mix of both discrete entropy and differential entropy.
14

---

Published as a conference paper at ICLR 2023
B PROOF OF SEMIRING
Below, we prove that the probability, log, entropy, log entropy, log reverse-KL. semirings defined in paper actual semirings according to Definition 2.2 Each proof is provided in four parts:
1. & forms commutative monoid with identity 0.
2. + forms a monoid with identity I.
3. & distributes over
4. 0 annihilates @
B.1 PROOF THAT THE PROBABILITY SEMIRING IS SEMIRING
Definition 2.5. The probability semiring can be defined as such: wo(e) - p(e) where p: E + [0 the probability function, +, x, 0 0, =1
Proof.
1. a
+ b b +
b 3 a
(b@c)
= a cb)
(a + +
= (a + b) 0c
a ① 0
a+0
2. a a (b 0)
(bxc)
(a x b) x
(a Cc
a
x 1 a.
180 a
1 x
= a.
3. a (boc)
x (b+c)
(a +) (xxe)
(a & ((
(b (b@c)
-b+c)xx xa
(bxa)
(boa)
4. a Ø
x 0
=0 0
0.
0
15

---

Published as a conference paper at ICLR 2023
0 X
O.
B.2 PROOF THAT THE LOG SEMIRING SEMIRING
Definition 2.7. The log semiring can defined such: we(e) log a @ log(e e²), a@b a. = -00, = 0.
Proof.
1. a aa@0 b
log(e" + eb) log(e" ea)
b @ a.
a + cb
@ log(e + E
log(e" to + e
log(e" + +e) Oc
(a + b) Bc
+ 0
log(e" + e )))
2. a a (b c)
(b c))
a +
(a &c
a
a 0
1
() +
a.
3. @ (b@c)
a + log(e tee
lob eato
= (a 8 b) (a
(b c) 8a
log(e" +c +a lta tta
(b a) @ (c a).
4. a
8--00 0c
-00
00
-00 +
00
0.
16

---

Published as a conference paper at ICLR 2023
B.3 PROOF THAT THE ENTROPY SEMIRING SEMIRING
Definition 2.9. The entropy semiring can be defined as such:
ur(e) (p(e), log
(a, b) @ (c,d) ta d),
(a. (c, d) (ac, ad + be),
0 (0, 0). I = (1.
Proof.
(a, @ d;
ca d)
actad+bb (c @ b).
e ((c. @ (f.g)) (f.
(a, 1 ( (c+
(a+t+
(a+c++d)((fg)))
((a (c.)))
(a, b) @
(a 0.
=(a,b) (a,
2. (a, b) @ (1c, d) @ (f.g))
(a, b) @ (cf.eg + df)
(acf, acg + adf +bef)
(ac, ad bc) @ (f.g) ((a 8 (c. 2 (f.g).
(a, I
(a x xxlaxxbxxx =a,
T ()
(1 x a, 0 x a + xx
-(a,b) b).
3. (a, & (1c. d) $ (f.g))
(a, b) 2 (c f.d+g)
(ac af ad ag
(ac, ad + bc) @ bf)
((a @ (c,d)) @ ((a @ (f,))).
(1c, d) e (f.g)) ((, (a
+c+ (
(ca fo. da g
(ca, (fa,
(a,b)) ((. 20 ())
4. (a, 0
(ax0,ax0+bx0 (0,0)
@ (a,
(0 x 0xxx0)x0
17

---

Published as a conference paper at ICLR 2023
0)
B.4 PROOF THAT THE LOG ENTROPY SEMIRING IS SEMIRING
Definition 4.1. The log entropy semiring be defined such:
p(e) w(e) = -ogp
(a, b) @ (c, d) (log(e" + log(e" + E²)),
(a, b) @ (c, d) = (a + c, log((e eata + clto)),
0 (-o0, 00). I (0, -00).
Proof.
1. (a. b) @ (c,
(log(e" +e log(e" + ed))
(log(e + log( + e²))
(c, d) @ (a,a)) b).
(a, b) @ (1c, d3 @ (S.g)
b) (logle +e =e),, logled -e9)
<log(e" + logle + +ed + +e"))
(logle" +e)), logg, + )) @ (fig)
((a,b) @ (c, @ (J.9).
(a, 6) e
(log(e" te + log(e" te )e) ba,
2. (a, b) @ ((c, di @ (f.g))
b) (c f, fllog(ests + edts))
ta flogeetttts + catd+s + b)ots))
la + a. lo + +cbte)) @ (f.g)
((a,b) @ (1) @ (J,g).
(a, b) @
(a + 0, Log(ea-oo + oto))
(a, b).
18 (a,
(0 + a log(e-ooton 0ota + t utb)]
b).
3. b) @ (1c. + (f.9))
(a, b) Ø (logle +el +el) logled + -e9)
(a logie- l, +eats etee + cbt)),
la au+ coggeet +elro)) @ f flggleta + elts))
((a,b) (c, @ ((a,b) (f.g)).
((c, d) e (fig)) & (a,b)
(log(e" + +,'), log(e" + +ee) @ (a.b)
(log(e + el)
le +elto edta + estay
loglett + tetta)) e (f loglee + testa))
((c, ()) @ ((f.g)) (o.
4. b)
la lo,log(e + e
00, -00)
18

---

Published as a conference paper at ICLR 2023
0 (a,b) b)
-00 + a, log(e
tota + e exonth)))
00, 00)
B.5 PROOF THAT THE LOG REVERSE-KE KL SEMIRING IS A SEMIRING
Definition 4.2. The log reverse-KL KL semiring can be defined as such:
wc(e) = (logp(e), log a(e), (c)) log log ((e)), log((a(() logg(e))))
(a, b, c, d) @ (f.g. h, i) = (log(e ),), log(e e)), log(e + e²)), log(e + et)), b,,) @ (f.g. h, +a+ f, b g, log(e h + ecto), log(e + edti))) 0 (-00, - ~c, 00, -00), = (0, 0, -00 -20).
Proof.
1. (a, ba, + hf.g.
'log(e" + +e'), Lo)), + + ),))),og((" + + ,e"),,og(e" + + +e'))
lolg(e + "ooggee ed) -S.ggh, e d,
(logle! + loglee
(a,b,c.d) d) @ (fS.g, @ m))
la,b,,, d) (log(e) + +e)), log(es + +e*), log(e" et), log(e' + eT)))
<logle" +et te)), + + +e*), + +e log), + + tct))
+ (log(e" log, + log lo"), + ee)) @ k,, (,
(l,,b, @ (S.g. i)) @ m))
ba,b,c,d df @0
(log(e" +e log(e" te te-a), log(e= te + lo + )0)) (a,b,c,
2. (a, la.be..d) d) @ (1f.g, @ (m),,l,m) (a,,,c,d) @ (J j.g kk log(estl + cltt),, log(estm + elttk) b logglcc te bthth + (ctott)) log + btitk + ((tgth))
-a+ g, + te log(ebti & + edto) & kik,l, = c(a,b,c, @ h. m))
ba, d) i (a + 0, b 0, log(e c log(ebmoo + elto)) b,,
I (a,b,c,
(0 (0+aa +,+ b, looto
ooth + loge ooth + cltd))
(a,b,c,d).
3. la,b,,,d d) & (VS.g,h, @ kj)
b,,b, (log(e) + ee), log(es + t logle" et), logle' em))
-a+loglee +e ed), +e),, b..bllogge + lo)),log(ebt + tebtl + tetts teeth), log btc + bebt +
+
la f.b g, log + etts) logle + dlo)) @ (a + j,b + k. log + logle, elthe)
-la,b,c,d) @ (f.g. h i)) @ (la,b, b. d) @ k,,)),
h, @ (j,k,l,,)) @ (a,b,c,d)
Alogle! + ee'),log(e + I, + et), log(e' + en)) @ (a, b,,
llogles log(e +et) +b, log(es tekta + t log(ect) + tottd mbbb)
(S ag + b, log(esto cb+) log(estd + c(t)) @ (j + a b, log(e +
19

---

Published as a conference paper at ICLR 2023
clt9), loglettd cm
= -(f.g. 1) @ (a, c, @ (m), & b,,))
4. (a, l,, 20 80
0., xe, log(e + ) logle + el-20))
00, 00, 00,- -00)
0.
(a, C. d)
100 a, -00 + b, logloee-oot + e-oot)), log(e- -o0t + c-ootd))
-00, 00, -00, -00)
20

---

Published as a conference paper at ICLR 2023
C EXTENDED EXAMPLES
U1
U4
03
Figure 2: DAG in Example 2.6
C.I DERIVATION FOR EXAMPLE 2.8
w(n)) w(22
0.
w(1g (w3) (w(vi) @ w(e))) @ (w((2)) & w(e2))
(0 + logp(ei) @ (0 + logg(ez))
logp(ei) @ logp(e2)
log (p(e)) p(22]]
w (w)) w(vz) wr(a))
log [p(e)) + ((2)) & loggles)
log (
((b)) w(v3) @ w(e4)
log (p(e)( p(e2)) & logr(e)
(( + +lee)))(e))
Ougg(s>
w(vs) @ w(vs)
log (e((ppe(((s)))) (es log((())))(s)) + bogpp(aa)(())pp())eea))))))) p(e2)((ea))
log + (a(on)e))))))))))))++++))++)) p(ee))))))
C.2 DERIVATION FOR EXAMPLE 2.10
w(v1) w(22)
(1,0).
ww3)) (w(vi) @ w(er)) o (w(22) & w(e2))
((1,0) @ (eee)) @ + (1 (ose)
(p(e1), log @ ple)
(p(e1) + p logl(e)
logp(e)) ple
w(4)) ) w(va) @ w(e3)
= + (e(el===n(illppo)oppox(++++)){d
ple))p(ss) (ospe))))e
w (ws) w(va) w(44)
(p(e1) + lo ppee) (laa))) (p(e1)p(e) + Meee) /agp://eeeeeeloiooooooeeeeeaaaaa loseep))))
w(vs) @ w(v))
(Cp(m), p(e )p(e)) log loge + log ((2)(())
+ p(e, )p(e)) log (p(e(()))(p)))) +(ee))) (o((e)))
21

---

Published as a conference paper at ICLR 2023
D VISUALIZATION FOR ENTROPY REGULARIZATION
short example picked at random from the Librispeech Test-Other set, which corresponds to the speech utterance "That's 's what believe young woman." " This example has an acoustic sequence length of 275 and text sequence length of 8, which corresponds to CTC lattice of size 275 by 17. Its waveform and log-mel spectrogram are respectively visualized in Fig. 3 and Fig.
Waveform
0.50
0.25
MWV
Icecondel
Figure 3: Visualization of waveform. The y-axis represents the amplitude of the signal, while the x x-axis represents the time.
Log Spectrogram
Rrequency Inz. melocala)
60
80
8.0
(xeconds)
Figure 4: Visualization of log-mel spectrogram. The y-axis represents the frequency, while the x-axis represents the time. The color gradients represent the magnitude (in decibels) of the signal.
Fig. 5 visualizes the CTC lattices for this example using the LSTM models trained in Section 5.1 to showcase the effects of entropy regularization. Interestingly, the lattice on the right, which cor- responds to higher alignment entropy, generally has higher "peaks" than the left lattice for both the blank labels and the acoustic regions corresponding to the non-blank labels. But these "peaks" are also wider, leading to an overall flatter and less peaky distribution. The visualization suggests that the surprising result of entropy regularization, which encourages the exploration of more alignment paths, is to concentrate, rather than diffuse, the probability mass around the right states in the lattice. In other words, instead of avoiding the local optima around the peaks, the model actually doubles down on them and widens the local exploration their vicinity.
While this is helpful average, it can also backfire. In Fig. 6 we visualize a pair of CTC lattices similar Fig. 5 but for an utterance where the entropy regularized model has a higher WER com- pared the NLL model. For this example, wider probability masses at some erroneous states like (Audio 55, Labels = 7). i.c. is actually more over-confident compared to the NLL model. This suggests that even though alignment entropy regularization is helpful on average, it might not sufficient for good model calibration since more local exploration around existing bad optima actually worsen the problem. Future work should investigate regularization mechanisms that encourage more global exploration further away from existing local optima in the alignment space.
that the entropy regularized model has higher and
22

---

Published as a conference paper at ICLR 2023
CTC Latfice with NLL
CTC Lattice with Entropy Regularization
20
20
30
50
60
60
90
90
100
0.4
120
130
130
0,3
160
160
370
170
190
0.2
200
200
210
21.
230
230
240
240
260
270
0.0 270
0.0
Labels
Labels
Figure 5: Visualization of CTC lattices. The left graph represents the CTC lattice trained with NLL, while the right graph represents the CTC lattice trained with entropy regularization. For both graphs, the y-axis represents the acoustic sequence, while the x-axis represents the text sequence. The even- numbered labels represent the blank token in CTC. The color gradients represent the liog-linearly scaled) probability mass of each CTC state.
Lattice
Lattice Regularization
22
55
0.5
321
110
0.4
121
0.4
337
154
154
176
187
187
220
264
0.2
220
242
233
233
275
236
286
300
Labels
Labels
Figure 6: Visualization of CTC lattices similar to Fig. 5 but with the probability mass erroneously concentrated in wrong states in the lattice, for example (Audio 55, Labels 7).
23


Reviews:
[{"review_id": "nyRwmkGQ5Yv", "reviewer": "Reviewer_NK1H", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a numerically stable and parallelizable version of the entropy semi-ring for ASR training. For numerical stability, the paper introduces log entropy semiring with $<\\log p(e), \\log(-p(e)\\log p(e))> $. For the knowledge distillation, which is based on the hard and soft labels, the paper proposes one more semi-ring called log reverse-KL semiring by concatenating log and log entropy semi-rings. This way, the paper tries to incorporate alignment information into the distillation loss. Experiments show the effectiveness of the method on Librispeech in two settings: (1) adding entropy regularization to CTC/RNNT LSTM/Conformer combinations reduce the WER especially when the base model is not very strong. (2) In the RNN-T distillation experiments add Libri-Light as the semisupervised data which are pseudo labeled by a strong supervised model, then a teacher is learned using a combination of these LibriLight and LibriSpeech data sets. The student model is trained on LibriLight using the teacher. The results show that the proposed distillation approach performs better than hard and soft distillation alone. Comparison with streaming models also show the success of the proposed approach. The paper will also be accompanied with an open-source implementation of the proposed approach.", "summary_of_the_review": "The paper brings the mathematically backed up idea of semi-rings back to ASR such that they can be used in CTC or RNNT systems. The paper provides both mathematical description and experimental evaluation of the proposed semirings. Experiments on entropy regularization and knowledge distillation show low enough WERs. The paper is well-written. Open source implementation will be provided and the paper should become reproducible. It is a good paper in general.  ", "strength_and_weaknesses": "Strengths:\n- The paper brings the mathematically backed up idea of semi-rings back to ASR such that they can be used in CTC or RNNT systems. \n- The paper is clearly written.\n- Experimental results show good WER performance in the Librispeech setting.    \n\nWeaknesses: \n- In Table 3, does any of the papers utilize the LibriLight data? If yes, it might be better to denote that in the table. Current semiring knowledge distillation proposal might be benefiting from that external data and the comparison might not be fair enough. \n- The appendix might be improved by giving some more details on the gradient back propagation over the semiring.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "nBVi9rDSB-a", "reviewer": "Reviewer_dC8C", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes use of entropy of speech-text alignment distribution as a regularizer during model training or as an additional supervision signal in the model distillation process.  As a regularizer the alignment entropy attempts to prevent the alignment distribution that’s induced without any supervision from becoming too concentrated on some, possibly erroneous, alignments.  And during model distillation the alignment entropy is intended to provide additional supervision to the student model.  The key idea in the proposed approach is to utilize dynamic programming with entropy semi-ring to efficiently compute the alignment entropy.  Authors propose and open-source a numerically stable and parallel implementation of CTC and RNN-T within the semi-ring framework.", "summary_of_the_review": "Paper makes a novel and significant contribution to enable use of alignment entropy as part of model training process.", "strength_and_weaknesses": "Pros:\n* A novel regularizer and distillation objective based on entropy of alignment distribution achieving state of the art performance on Librispeech in streaming scenario.\n* Open-source stable and parallel implementation for CTC and RNN-T\n\nCons:\nN/A\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "47CzqGjLyn", "reviewer": "Reviewer_3ceX", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper authors propose to leverage entropy semiring and alignment entropy to improve the performance of neural speech recognition, via regularization or distillation. Experimental results show its effectiveness. There are also open-source contributions based on this work.", "summary_of_the_review": "The methodology presented is technical sound, with convincing experimental design and results. Open-source implementation makes it easier for speech community to leverage this work. I think the experimental section could be made stronger by adding more details to justify design and results, including:\n\n1. Adding std to those results in Table 1-3 to show if improvements are statistically significant.\n2. How to determine model hyperparameters, e.g. those listed in the \"Experimental Setup\" paragraphs in Sec 5.1 & 5.2.\n3. As shown in Table 3, prior work has different model sizes. Do authors also experiment with smaller models (e.g. similar as 30M or 80M #Params)?", "strength_and_weaknesses": "Strength: The idea of regularization or distillation based on alignment entropy is technically sound. Overall, this paper is well structured, with detailed background introduction on semiring. Experiments are well-designed for validation purpose. Open-source contribution with this work makes it easier for speech community to experiment with and integrate this approach. \n\nWeakness: I think experimental section could be made stronger with more details added to explain design and results. See questions in the summary section below.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1OCp1xyCHFR", "reviewer": "Reviewer_jsaa", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new speech recognition formulation based on the entropy semiring, which is used as an entropy regularization and knowledge distillation. First, the paper motivates hard alignment-based speech recognition issues based on overconfidence and the solution based on entropy regularization. Then, the paper provides an elegant and easy-to-understand formulation of the semiring from the basic one to their novel entropy semiring by caring for the backpropagation. The experiments also show the expected result of using entropy regularization and knowledge distillation experiments for the well-known Librispeech benchmarks with state-of-the-art performance in a streaming setup. \n\n", "summary_of_the_review": "The paper's novelty, clearness, experimental effectiveness, and reproducibility significantly contribute to the machine-learning community. I strongly recommend this paper be accepted.\n\nOther suggestions\n- It would be better to provide some more concrete examples for the semiring explanation (e.g., Example 2.6 corresponds to the forward computation, right? We can provide such an example)\n- Can we visualize whether the overconfidence is mitigated or not in Section 5.1? I think such analysis (visualization) strengthens the claim of this paper.\n- Section 5.1: It might significantly improve the performance in the out-of-domain conditions (e.g., noisy speech recognition, multilingual speech recognition). I would recommend the authors apply such tasks to prove further the effectiveness of mitigating overconfidence by the proposed method.\n", "strength_and_weaknesses": "Strength\n\n- clear descriptions of each semiring operations\n- novel entropy semiring formulation by considering the backpropagation capability and efficiency.\n- reasonable experimental results showing the mitigation of overconfidence and the knowledge distillation\n- archives the state-of-the-art performance in the Librispeech streaming setup\n\nWeaknesses\n\n- it could have more discussions about other applications than ASR (although they are briefly mentioned in the conclusion section).\n- lack of the analysis\n- there are some duplicated descriptions (e.g., the first paragraph in Section 3 and the first paragraph in Section 3 are highly overlapped with Section 1). These parts can be improved. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster