Paper: Contextual Biasing with the Knuth-Morris-Pratt Matching Algorithm (ICLR-2024)

Reviewer comment (Experiments):
The main objective of the paper is to build a contextual biasing system that is efficient to decode on large-scale parallelizable infrastructure such as TPUs. However, in the introduction and the experiments, the application of the method is for recognizing contact names for voice assistants. In my understanding, such voice assistants are commonly placed on the edge device, which does not usually have built-in TPUs. As such, it is hard to see what would be the impact of the proposed method from a decoding efficiency perspective.

Author response:
## **Practicality of biasing on TPU/GPU (or in general high-performance parallel computing devices)**

On-TPU/GPU biasing is practical for voice assistants, because server ASR models can be run on them to enjoy high throughput via batching. And while not all on-device ASR models have access to TPU/GPUs, modern devices like pixel phones are equipped with high performance chips, and such devices will become increasingly popular and available as their cost goes down.

Reviewer comment (Presentation):
The authors rely too much on Algorithm blocks to present their method (there are 5 in total including the appendices), which breaks the flow of reading and makes the paper hard to parse. It may be beneficial to release open-source code for the details of the algorithm and use more space to discuss the algorithms themselves, their connections with other contextual biasing methods, and their advantages/limitations.

Author response:
We will incorporate your suggestions on paper presentation in a later version, especially the connection with other contextual biasing methods, and a discussion on the currently available specialized libraries.

Reviewer comment (Theory):
There are several linear-time algorithms for pattern matching such as Rabin-Karp, Boyer-Moore, etc. It would be useful to include a discussion about why KMP is most appropriate for the task.

Author response:
## **Other pattern matching algorithms**

Thank you for pointing out the other pattern matching algorithms (Rabin-Karp and Boyer-Moore). KMP is a classical algorithm covered in standard textbooks, and has the best worst-case complexity (for a single search pattern). It is possible that other methods offer better trade-offs in certain scenarios and may be investigated in future work.

Reviewer comment (Experiments):
With increased B (number of biasing phrases), the result gets worse. First, to clarify: The test dataset is designed such that the word from reference transcription is always in the biasing phrases? Or otherwise, why would it get worse with increased B? Additionally, in practice, how big would B be? How should the set of biasing phrases be updated? Wouldn't it make more sense to test in a way that is closer to how this is actually used in production?

Author response:
(Just realized that we have not addressed your question on B.)

The test dataset is designed such that the word from reference transcription is in the biasing phrases, except for *anti-biasing* where all provided biasing phrases are distractors. In general, we want to improve WER when ground truth is provided, and not to degrade WER when it is not. The larger the B, the worse the WER is because there are more possibilities to bias towards, and therefore more chances to confuse the ASR model.

The actual number of B depends on the specific application scenario. In the case of on-device call contact biasing, the number of biasing phrases is the number of contacts in phone book, which is typically few hundreds (as in our Contact-Tag test set). In the case of biasing for geographical locations, the number of biasing phrases could be quite a few thousands or more.

Reviewer comment (Novelty):
The paper lacks adequate references to the related work in the area of the contextual biasing for ASR Models. I have added some relevant citations.

Author response:
We thank the reviewer for pointing out the references and we will incorporate them into a later version. Your other questions have been addressed in "Rebuttal to common concerns".

Reviewer comment (Theory):
In this work, the bonus score is added each time the match happens, how do you address the scenario where prefix has matched but suffix won't match and the hypothesis are still carrying extra weight provided during the biasing? E.g, if the dictionary item is TWIN and the actual audio has TWENTE, but the hypothesis is preferring TWIN (and/or its continuations) due to additional biasing.

Author response:
## **Regarding your question 1**

Assume the dictionary contains only one item “twin”. And assume that the model uses beam_size=1, and has already output “_tw”, and thus a bonus for “_tw” was added to the hyp.

In the next step, the ASR model considers two possible expansions “en” and “in”.

- “in” leads to longer matching of the dictionary item “twin”, so the proposal “in” receives a bonus.

- “en” leads to non-matching of any dictionary item, and therefore it also has to cancel the bonus received by “_tw”. The canceling part is implied by our potential function, dependent on the longest partial matching length (which is 0 after accepting "en"), and the bonus being the difference between potentials before and after accepting the token; see Section 2.2.

Therefore it boils down to which is higher, score(“in”) + bonus(“in”) versus score(“en”) - bonus(“_tw”), and the token with higher final score will be accepted and the other is pruned. 

It may be the case that the ASR score for “en” is much higher than that of “in”, and “twen” becomes the top hypothesis, and search continues with a total biasing bonus of 0 (due to the canceling of bonus).