RuntimeError: start (2) + length (1) exceeds dimension size (2) when trying to run Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**

Hello, I was trying to run Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb but I get an error when running the final cell in Jupyter

[Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/fawazahamedshaik/Downloads/Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb) Cell 13 line 8
76 cache_pre_encode = processed_signal[:, :, -pre_encode_cache_size:]
78 with torch.no_grad():
79 (
80 pred_out_stream,
81 transcribed_texts,
82 cache_last_channel,
83 cache_last_time,
84 cache_last_channel_len,
85 previous_hypotheses,
---> 86 ) = asr_model.conformer_stream_step(
87 processed_signal=processed_signal,
88 processed_signal_length=processed_signal_length,
89 cache_last_channel=cache_last_channel,
90 cache_last_time=cache_last_time,
91 cache_last_channel_len=cache_last_channel_len,
...
--> 400 f = x.narrow(dim=0, start=time_idx, length=1)
402 # Setup exit flags and counter
403 not_blank = True

RuntimeError: start (2) + length (1) exceeds dimension size (2).

Any advice on how to mitigate this error?

I am running this in M2 MacBook Pro


**Environment overview (please complete the following information)**

 - Environment location: local
 - Method of NeMo install: installed using pip install git+https://github.com/NVIDIA/NeMo.git


**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version - macOS 13.3.1
- PyTorch version - pytorch-lightning  2.2.1
- Python version - 3.10.0

**Additional context**

Add any other context about the problem here.
running on m2 chip


Hi @tempops thanks for sharing this issue. I am able to reproduce it in some cases. Not sure yet why it occurs. Could you please share what NeMo version you are using, and the exact command you used to install NeMo?

As a quick fix, you can add an extra line in one of the cells as I show below. I believe that will fix the error.
```
    ...
    asr_model.encoder.set_default_att_context_size([left_context_size, int(lookahead_size / ENCODER_STEP_LENGTH)])

# new line
asr_model.encoder.setup_streaming_params()

# make sure we use the specified decoder_type
asr_model.change_decoding_strategy(decoder_type=decoder_type)
...
```

Let me know if it helps.

cc @VahidooX 

Hello @erastorgueva-nv thanks for the response!

I followed your code and got it to work, another thing to mention is I updated my Nemo_toolkit package version to latest.

However when I try decoding through microphone it's very glitchy, as I see that it misses transcribing bits of spoken audio, I assumed that this might be due to the lookahead size so I tested with all available sizes 0, 80, 480, 1040 but still I see inconsistency in the output.

I am using the "stt_en_fastconformer_hybrid_large_streaming_multi" model, and using nemo-asr  0.9.0 and nemo_toolkit                             2.0.0rc0 versions

Is it because I am running it on an M2 Mac? will the performance be better with a GPU? I tried running the same in a windows laptop which has a Nvidia GPU (not sure if I am using it during inference though) and it runs way more smoother and lesser inconsistencies but still I see some of the audio not being transcribed (lot lesser error than running in Mac). I also dont have ffmpeg installed in the windows laptop so not sure if that might be causing the issue.

nemo version used in windows = Nemo_toolkit 1.23.0rc0

Is the choppy transcription due to the chunk/cache aware nature of model itself? or am I missing something and not running it properly?



This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

Hi @tempops , this is unexpected to me. I have run the models on an Intel 2019 Mac and the transcriptions don't seem to miss anything. Maybe if I mumble some audio might be missed sometimes.

"using nemo-asr 0.9.0" is a bit odd, what do you mean exactly?

Hello,

Thank you for the response,

by nemo-asr I was using https://pypi.org/project/nemo-asr/. I am assuming it's not needed as you mention that it is odd.

I am running this on Mac M2 Chip, could it be because I am not running it on intel.

Can you share the exact specification (Packages and Hardware, Software specs) of your run so that I can recreate it, and whether you are using a GPU. 

I am running the stt_en_fastconformer_hybrid_large_streaming_multi model with 1040 lookahead size and 'rnnt' decoding strategy. 

Thank you!


Hi, for sure you should not be using `nemo-asr`, that package is super old. Thanks for letting me know it exists, I will see if we can remove it.
I will try to get back to you regarding exact specs. In the meantime, if you have time you can try making a fresh environment without `nemo-asr`.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.