Repository: NVIDIA/NeMo

Issue:
eval_beamsearch_ngram.py fails (pack_hypotheses invalid data type 'str')
**Describe the bug**

I trained QuartzNet acoustic model + KenLM 2-ngram model, now I run eval_beamsearch_ngram.py to evaluate it. 

It seems like beam_search_func in BeamSearchDecoderWithLM outputs tuples (confidence, text) (the doc agrees), so cand.y_sequence is a resulting string. 

but later on the function pack_hypothesis in ctc_beam_decoding does "cand.y_sequence = torch.tensor(cand.y_sequence, dtype=torch.long)" and expects it to be int labels, so I get an error. Meanwhile greedy decoding works perfectly.

**Steps/Code to reproduce bug**

I am just following the steps here https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/asr/asr_language_modeling.html

python eval_beamsearch_ngram.py nemo_model_file=*quartznet_model_from_default_config* \
           input_manifest="" \
           kenlm_model_file=*scorer from train_kenlm* \
           beam_width=[500] \
           beam_alpha=[0.82] \
           beam_beta=[0.87] \
           preds_output_folder= \
           probs_cache_file=null \
           decoding_mode=beamsearch_ngram


**Expected behavior**

I expected the eval_beamsearch_ngram.py script work with default QuartzNet and default KenLM trained model. I am not exactly sure if I am not specifying the correct params for it. 

**Environment overview (please complete the following information)**

 - Environment location: linux
 - Method of NeMo install: pip install from tutorial

**Environment details**

Otherwise, please provide:
- OS version: ubuntu 18.04.5 LTS (Bionic Beaver)
- PyTorch version 1.13.1
- Python version 3.8.12

**Additional context**

nemo toolkit 1.16.0rc0
ctc-decoders version is 1.1




Discussion:
Hmm I didn't test this with QuartzNet but it seems with char based model it is not correctly returning a list of integer IDs but a str. I will take a look next week 

Resolving patch:
diff --git a/nemo/collections/asr/parts/submodules/ctc_beam_decoding.py b/nemo/collections/asr/parts/submodules/ctc_beam_decoding.py
--- a/nemo/collections/asr/parts/submodules/ctc_beam_decoding.py
+++ b/nemo/collections/asr/parts/submodules/ctc_beam_decoding.py
@@ -98,6 +98,10 @@ def __init__(self, blank_id: int, beam_size: int):
         self.decoding_type = None
         self.tokenizer = None
 
+        # Utility maps for vocabulary
+        self.vocab_index_map = None
+        self.index_vocab_map = None
+
         # Internal variable, used to prevent double reduction of consecutive tokens (ctc collapse)
         self.override_fold_consecutive_value = None
 
@@ -110,6 +114,8 @@ def set_vocabulary(self, vocab: List[str]):
                 Note that this vocabulary must NOT contain the "BLANK" token.
         """
         self.vocab = vocab
+        self.vocab_index_map = {v: i for i, v in enumerate(vocab)}
+        self.index_vocab_map = {i: v for i, v in enumerate(vocab)}
 
     def set_decoding_type(self, decoding_type: str):
         """
@@ -352,7 +358,8 @@ def default_beam_search(
                 if self.decoding_type == 'subword':
                     pred_token_ids = [ord(c) - self.token_offset for c in candidate[1]]
                 else:
-                    pred_token_ids = candidate[1]
+                    # Char models
+                    pred_token_ids = [self.vocab_index_map[c] for c in candidate[1]]
 
                 # We preserve the token ids and the score for this hypothesis
                 hypothesis.y_sequence = pred_token_ids
@@ -444,7 +451,7 @@ def _pyctcdecode_beam_search(
                         raise ValueError("Vocab must be provided for character decoding. Use set_vocab().")
 
                     chars = list(candidate[0])
-                    pred_token_ids = [self.vocab[c] for c in chars]
+                    pred_token_ids = [self.vocab_index_map[c] for c in chars]
 
                 hypothesis.y_sequence = pred_token_ids
                 hypothesis.text = candidate[0]  # text


Test patch: