Paper: Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework (ICLR-2024)

Reviewer comment (Experiments):
While the latency reduction part presents extensive experimental results (various training data sizes, comparison, ablation, ...), there are not many results on minimum WER training. Furthermore, the gain from minimum WER training is marginal.

Author response:
**While the latency reduction part presents extensive experimental results, there are fewer results on minimum WER training, and their gain is marginal.**

The evidence for the successful application of AWP to the low latency application is stronger and wider. Yet, although more modest, the WER part presents 2 model architectures, and another method for optimizing mWER was added to table 3 for additional reference (see the ‘common response’ part). The gain in this part is moderate compared to prior work.  Graves & Jaitly, 2014 reported a relative improvement ranging from 3.7% to 9% on the Wall Street Journal dataset. Prabhavalkar et al,. 2018 reported an improvement ranging between 4.1% to 7.4%, and [1] reported a 5% improvement, both on non-public datasets. We believe that although the gains for the latency application are more prominent than the WER application, the relative improvement in the WER application demonstrates the versatility and the generality of AWP.

Reviewer comment (Experiments):
There seems to be room for improvement; for example, how about increasing the number of shifted frames (tokens) instead of selecting just one? How about applying AWP together with Trim-Tail? I appreciate the simplicity of the proposed method, but I am also curious about the limitations of this method.

Author response:
**Can results be improved by increasing the number of shifted frames (tokens), or by applying AWP together with Trim-Tail, for example?**

As demonstrated in table 2, AWP achieves better results than the other methods both in latency and in accuracy, including reaching a negative DL. Increasing the number of shifted frames yielded a similar trend, but our empirical findings suggest that gradual changes were favored. We added appendix C to discuss the variations of the property function, we kindly refer the reviewer to the ‘common response’. 
Exploring the combination of AWP with other methods like Trim-Tail can be very promising. However, it can be expected that the trade off between latency and accuracy may still play an important role, as decreasing the latency even further is expected to increase the WER.

Reviewer comment (Presentation):
What is WSR in Figures 6 and 7? Is it (100 – WER)?

Author response:
**WSR is figured 6 and 7**

WSR is the word success rate, 100-WER. We modified figure 6 to present WER, and changed the caption of figure 7.

Reviewer comment (Reproducibility):
It seems that AWP needs random sampling at each training step. How many alignments (N) do you sample for each step? How does AWP affect the overall training time/resource usage?

Author response:
**How many alignments (N) do you sample for each step? How does AWP affect the overall training time/resource usage?**
 
We thank the reviewer for the question, we added this information to appendix A. Under the low latency setting, we set N=5 and for the WER setting we set N=10, based on empirical observations. 
As mentioned, sampling occurs every training step, and training with AWP is, empirically, 2-3 times slower than training a baseline model.

Reviewer comment (Experiments):
absence of comparison of proposed method with prior works and baselines (e.g. if we introduce reweighing of alignments directly in the CTC loss)

Author response:
**Absence of comparison with prior works**

For the latency application, we’ve conducted an extensive comparison to prior works. To ensure a fair comparison, we implemented or adjusted public code to fit our setting. Additionally, we enhanced the text with an additional mWER optimization for comparison, we kindly refer the reviewer to the ‘common response’ for details.

Reviewer comment (Reproducibility):
missing details on how exactly the sampling of the word to be corrected is implemented, as it could be that no word is available for substitution. Why WER and not CER as language model could fix the errors? Is language model used in this process?

Author response:
**Missing details on the implementation. Why WER and not CER? Is language model (LM) used?**
We optimize WER rather than CER as WER is the primary metric for evaluating ASR systems. Furthermore, CTC is more closely related to CER than WER, given its indifference to whether errors are localized within a single word or distributed across multiple words (see fig 2.b in the text). In response to the reviewer’s comment, we added the code for WER optimization to the supplementary material. Concerning the LM, our implementation uses an LM in the beam search decoding (we kindly refer the review to appendix A1).

Reviewer comment (Experiments):
I believe that sampling alignments is not the right / optimal way: we could consider optimization of the top-N alignments instead, as otherwise we spend time on optimization low probability alignments.

Author response:
**Sampling vs. top-N alignments**

Empirically, we found no evidence in favour of top-N alignments. We elaborated on this issue in the ‘common response’, we kindly refer the reviewer to this section.

Reviewer comment (Evaluation):
A bunch of models in Table 2 (on latency empirical results) are not comparable: either latency should be fixed and WER is compared or vice versa. Right now it is hard to make any conclusions from Table 1 due to different values for latency and WER of different models.

Author response:
**Comparing models in table 2**

Each of the top 2 sections in table 2, along with the Conformer section, exhibit comparability. It presents an offline model, an online model and an online model + AWP, and shows clear trend: the offline model is superior in terms of WER but also has the highest TL, the online model improves the TL on the expense of WER, and the online + AWP improves the TL even further in the cost of a WER. 
The third section presents these 3 models, alongside additional losses or input manipulation. We can see that AWP performs better in both DL and WER.

Reviewer comment (Evaluation):
I found it overall hard to formulate the proper reweighing between different alignments, rather than simple way of controlling the latency by restricting context or optimising directly mWER. It is not clear why proposed way of sampling alignments is sufficient or significantly beneficial. Overall, reported results are only marginal.

Author response:
**I found it overall hard to formulate the proper reweighing between different alignments, rather than simple way of controlling the latency by restricting context or optimising directly mWER. Reported results are marginal.**

We will endeavor to respond to the best of our understanding. AWP only requires a definition of a property function, which given an alignment, yields an improved alignment w.r.t to the property. Thus, it does not explicitly mandate the user to formulate a reweighting scheme. As for the advantages of AWP over alternatives - We discuss in the paper that context restriction allows control of the latency, but it also results in a drift (DL) in the online model. With AWP we were able to reduce this drift at the expense of WER. In table 2 we show that AWP can have a negative drift. Overall, we present compelling evidence that AWP can be successfully applied to low latency applications across, and yield moderate gains in WER optimization.

Reviewer comment (Writing):
"CTC posteriors tend to be peaky (Zeyer et al., 2021; Tian et al., 2022), and hence the posterior of one specific alignment is dominant over the others." I would smooth this formulation a bit, as likely several tokens are dominant for each time frame, and thus set of alignments (few) are dominant, not only one as discussed in a bit in Likhomanenko, T., Collobert, R., Jaitly, N., & Bengio, S. (2023, February). Continuous Soft Pseudo-Labeling in ASR. In Proceedings on (pp. 66-84). PMLR.

Author response:
**CTC posteriors**

We agree, text was updated accordingly.

Reviewer comment (Experiments):
what will happen if we use several tokens removal for the latency function instead of only 1 token? is it improving latency?

Author response:
**Shifting several tokens**

We kindly refer the reviewer to the ‘general response’ part, where we elaborated on variations of the property function.

Reviewer comment (Experiments):
"35K hours curated from LibriVox" is it multilingual or English only? If multilingual, why not English only as then another confound factor is introduced?

Author response:
**35K hours curated from LibriVox**

English only.

Reviewer comment (Experiments):
Throughout the text it is not clear where LM is used or not in the reported numbers.

Author response:
**LM usage**

LM is used for decoding. We provide details in Appendix A1.

Reviewer comment (Presentation):
What is WSR abbreviation (I could not find this notation in the text)? Figure 6 is hard to parse in the current form.

Author response:
**WSR**

WSR is the word success rate. Figure 6 now shows WER.

Reviewer comment (Experiments):
Why are multilingual wav2vec used for experiments? why not English only?

Author response:
**multilingual w2v**

We aimed for the base size model, and as far as we know, this model was trained on the largest training set for such a model.

Reviewer comment (Evaluation):
what is the relation between Table 2 and Table 4 for the prior methods? I see that Table 4 has better results in WER than in Table 2.

Author response:
**The relation between Tables 2 and 4**

Both present other applications to mitigate the DL. To compare table 4 to AWP, we see that AWP achieves the best working point in terms of both DL and WER. Other approaches may yield better WER or latency, but not simultaneously both.

Reviewer comment (Experiments):
why are features computed with "32ms window with a stride of 16ms"? This is really very non-standard Mel filter banks extraction for ASR models.

Author response:
**32ms window, stride of 16ms**

 It is optimized for STFT that works in multiples of 2.

Reviewer comment (Experiments):
improvements for AWP in min WER task is marginal having std 0.03 for clean set (while for latency task AWP really has significant improvement). I still would like to see greedy decoding results for the final version of the paper, as maybe it improves more, but doesn't make significant improvement after beam-search decoding with LM (as LM knowledge is transferred into acoustic model). This can give more insights in future.

Author response:
> Could you provide the exact typical std value you get? And this should be included in the revision, e.g. in Appendix if you still don't want to overload the main text. But for readers familiar with the dataset 0.1 difference will be insignificant.

We are not familiar with the 0.1 difference for the LibriSpeech Test Clean dataset. Based on our experiments with several seeds across several settings, we received that the std was no more than 0.03.

Reviewer comment (Experiments):
For the minimum word error rate application, Prabhavalkar et al. 2018 found that using the n-best beam search hypotheses is more effective than the sampling-based approach. This paper should do a similar comparison whether it should compute the property function in the n-best alignments instead of sampled alignments.

Author response:
We thank the reviewer for this comment. We have conducted experiments with both sampling and n-best BS hypotheses, which didn’t indicate that n-best BS is superior to the sampling method. Moreover, running a BS decoder at every training step requires more resources and training time. We kindly refer the reviewer to the ‘common response’ for elaborated explanation and a speculation as to why n-best BS method is not superior in this case.

Reviewer comment (Experiments):
Latency and WER optimizations typically compete with each other. It would be great to utilize the proposed framework to optimize these two properties jointly with a single property function to see if they can be balanced better together, and also see how effective and generalizable the framework is.

Author response:
WER and Latency were two property examples for AWP. We optimized each property separately in order to compare with previous results. Yet, the simplicity of AWP allows us to combine the losses together, or to use a single property function that improves both properties simultaneously, and test how the two properties compete with each other. We added this comment to the discussion in section 5.

Reviewer comment (Experiments):
How sensitive is the optimization to the specific choice of the property function? E.g. the currently designed latency function only allows one time step faster, and property function for the minimum WER application only allows one word error reduction. Are these choices made in order to stabilize the training, or actually they can be relaxed to allow more changes as well? The paper should compare more different property function choices for the same specific application.

Author response:
We experimented with multiple implementations of the property functions, and eventually reported the simplest (and evidently the most effective) implementation. Empirically, we found that other implementations worked as well. We added appendix C in the paper to shed more light on that matter. We kindly refer the reviewer to the ‘common response’ for more details on the different variations of the property function.

Reviewer comment (Experiments):
Adding the latency optimization to an offline Conformer model doesn't make much sense, since the full-context model is not used in a streaming fashion and it has to wait for the entire sentence to come first, which by itself already has a much higher latency. Conformer can be implemented in a streaming manner as well by just using the left context, which can be optimized for edge devices as well. The latency experiment should be conducted on an online Conformer.

Author response:
We appreciate the reviewer's feedback. The Conformer experiments were not fully matured by the ICLR deadline. We chose to present the offline model with AWP as a proof of concept, assuming success with an offline model implies potential success with an online model. To adapt the offline model to an online setting, at inference we restricted its context (both right and left) so it works in a streaming manner. We updated table 2 with the online Conformer, and show that although it wasn’t trained as an online model, but inferred as one, the performance of the Conformer with AWP shows a significant DL reduction and only a mild reduction in WER.

Reviewer comment (Experiments):
Using "start epoch" as a tunable hyperparameter to control the balance between latency and quality is a bit strange. How transferable is the optimal start epoch to different learning schedule, model architectures and data?

Author response:
Figure 6 right indicates that tuning alpha is a better practice for balancing between latency and accuracy. However, figure 6 left shows that tuning start-epoch allows some balancing, as there’s a ~11% WER deterioration if applied after 0.9 epochs instead of 16.7. Empirically we found that applying AWP works best after CTC training loss reaches 80-90% accuracy, which may vary across different settings (i.e. dataset, etc). We kindly refer the reviewer to the ‘common response’  for more details.

Reviewer comment (Writing):
In Figure 6: What is WSR? It is not a standard metric and it is not defined anywhere.

Author response:
WSR is word success rate, 100-WER. We changed figure 6 to present WER.