Repository: nvidia-riva/riva-asrlib-decoder (language: python)

Issue:
Accuracy difference
Hey Hi,
            I am trying to use Riva decoder as a substitution for k2 decoder. With riva decoder I am not getting the same accuracy as that of k2 decoder. There are lot more deletions, at least 12%.  Experimented with different hypereparamters like acoustic_scale, max_active_states, but result seems to be not changing much. I have tried with the different topologies as well(eesen, compact), its the same case with all of them. Can you please help in this regard. 


Discussion / hints:
@kbramhendra Thanks for the report. I've passed this on to our team internally.
@kbramhendra what value have you set for "max_expand"?

We have it set to 10 here:

https://github.com/nvidia-riva/riva-asrlib-decoder/blob/77f08769736aafabd8422a05e6692b81ffb5a618/src/riva/asrlib/decoder/test_graph_construction.py#L170

I have noticed that disabling this (setting it to 0) does improve WER. However, I am not certain where it is your issue based on what you have told me.

This is to account for an explosion in the state space of a depth first search in "word alignment" algorithm that can happen in rare circumstances.

Now, word alignment isn't necessary, strictly speaking, so I could consider disabling it, but I am still looking into what the "right" solution is here.

Fix patch:
diff --git a/.gitmodules b/.gitmodules
index 251d074..d727864 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -1,6 +1,6 @@
 [submodule "third_party/kaldi"]
 	path = third_party/kaldi
-	url = https://gitlab-master.nvidia.com/belgarten/kaldi.git
+	url = https://github.com/galv/kaldi.git
 [submodule "third_party/pybind11"]
 	path = third_party/pybind11
 	url = https://github.com/pybind/pybind11.git
diff --git a/third_party/kaldi b/third_party/kaldi
index 6e633c9..b3163d3 160000
--- a/third_party/kaldi
+++ b/third_party/kaldi
@@ -1,1 +1,1 @@
-Subproject commit 6e633c9a3885be849970f76ae0f032a30e6effe5
+Subproject commit b3163d3a2e40287c4fc39fbabc9c05c87e974f4c


Test patch:
diff --git a/src/riva/asrlib/decoder/test_graph_construction.py b/src/riva/asrlib/decoder/test_graph_construction.py
index 3ddc2cd..f395703 100644
--- a/src/riva/asrlib/decoder/test_graph_construction.py
+++ b/src/riva/asrlib/decoder/test_graph_construction.py
@@ -157,9 +157,6 @@ def run_decoder(self, graph_path: str):
 
         config = BatchedMappedDecoderCudaConfig()
         config.n_input_per_chunk = 50
-        config.online_opts.lattice_postprocessor_opts.word_boundary_rxfilename = str(
-            os.path.join(graph_path, "graph/graph_ctc_3-gram.pruned.3e-7/phones/word_boundary.int")
-        )
         config.online_opts.decoder_opts.default_beam = 17.0
         config.online_opts.decoder_opts.lattice_beam = 8.0
         config.online_opts.decoder_opts.max_active = 7000
@@ -167,7 +164,6 @@ def run_decoder(self, graph_path: str):
         config.online_opts.max_batch_size = 400
         config.online_opts.num_channels = 800
         config.online_opts.frame_shift_seconds = 0.03
-        config.online_opts.lattice_postprocessor_opts.max_expand = 10
         decoder = BatchedMappedDecoderCuda(
             config,
             os.path.join(graph_path, "graph/graph_ctc_3-gram.pruned.3e-7/TLG.fst"),