fix: clamp keep input size in update_cache for causal conv
State: closed | Pull request: yes
Labels: name: stale

name: ASR

# What does this PR do ?

Sometimes in CausalConv1D.update_cache input_x_keep ends up having no frames (ie size of [M, N, 0]). Make sure that the we have at least one frame.

**Collection**: asr

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.
@VahidooX 

# Additional Information
To reproduce the original issue in main run:
```
#!/bin/bash
python examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
    --asr_model=stt_en_conformer_ctc_small \
    --chunk_size=100 \
    --shift_size=50 \
    --left_chunks=2 \
    --online_normalization \
    --manifest_file=/datasets/ls_test_other/transcripts.local.json \
    --batch_size=16 \
    --compare_vs_offline \
    --use_amp \
    --debug_mode
```

Error output:
```
...
Traceback (most recent call last):
  File "examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py", line 393, in <module>
    main()
  File "examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py", line 349, in main
    streaming_tran, offline_tran = perform_streaming(
  File "examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py", line 154, in perform_streaming
    ) = asr_model.conformer_stream_step(
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/mixins/mixins.py", line 475, in conformer_stream_step
    (encoded, encoded_len, cache_last_channel_next, cache_last_time_next) = self.encoder.cache_aware_stream_step(
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/mixins/streaming.py", line 61, in cache_aware_stream_step
    encoder_output = self(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/core/classes/common.py", line 1087, in __call__
    outputs = wrapped(*args, **kwargs)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/modules/conformer_encoder.py", line 471, in forward
    audio_signal = layer(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/submodules/conformer_modules.py", line 191, in forward
    x = self.conv(x, pad_mask=pad_mask, cache=cache_last_time, cache_next=cache_last_time_next)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/submodules/conformer_modules.py", line 350, in forward
    x = self.depthwise_conv(x, cache=cache, cache_next=cache_next)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/submodules/causal_convs.py", line 162, in forward
    x = self.update_cache(x, cache=cache, cache_next=cache_next)
  File "/home/grclark/code/NeMo.git/streaming-conformer/nemo/collections/asr/parts/submodules/causal_convs.py", line 158, in update_cache
    cache_next[self._cache_id, :, :, -cache_keep_size:] = input_x_kept[:, :, -cache_keep_size:]
RuntimeError: The expanded size of the tensor (1) must match the existing size (0) at non-singleton dimension 2.  Target sizes: [16, 176, 1].  Tensor sizes: [16, 176, 0]
```