support for ndarray and tensor decoding for RNNT based ASR models
State: closed | Pull request: no

At first by observing recent feature updates( https://github.com/NVIDIA/NeMo/pull/8167#issue-2079986486 ) I thought RNNT based models could also receive arrays as inputs.
The recent update in source code for CTC model seems to be updated to accept inputs as ndarrays and tensors (https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/ctc_models.py)
```python
    def transcribe(
        self,
        audio: Union[str, List[str], torch.Tensor, np.ndarray],
        batch_size: int = 4,
        return_hypotheses: bool = False,
        num_workers: int = 0,
        channel_selector: Optional[ChannelSelectorType] = None,
        augmentor: DictConfig = None,
        verbose: bool = True,
        override_config: Optional[TranscribeConfig] = None,
    ) -> TranscriptionReturnType:
```


However, I found RNNT based models are unable to recieve ndarrays or torch tensors 
```python
    def transcribe(
        self,
        audio: List[str],
        batch_size: int = 4,
        return_hypotheses: bool = False,
        partial_hypothesis: Optional[List['Hypothesis']] = None,
        num_workers: int = 0,
        channel_selector: Optional[ChannelSelectorType] = None,
        augmentor: DictConfig = None,
        verbose: bool = True,
        override_config: Optional[TranscribeConfig] = None,
    ) -> TranscriptionReturnType:
```

I want to know if the direct transcription of arrays and tensors for RNNT based models will be available in the next release.

@titu1994 

The docstring isn't updated, but yes main branch supports rnnt as of right now - https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/rnnt_models.py#L274

> The docstring isn't updated, but yes main branch supports rnnt as of right now - https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/rnnt_models.py#L274


Thanks for the quick reply, but if we check the input arguments right now,
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/rnnt_models.py#L237

```python
   @torch.no_grad()
    def transcribe(
        self,
        audio: List[str],
        batch_size: int = 4,
        return_hypotheses: bool = False,
        partial_hypothesis: Optional[List['Hypothesis']] = None,
        num_workers: int = 0,
        channel_selector: Optional[ChannelSelectorType] = None,
        augmentor: DictConfig = None,
        verbose: bool = True,
        override_config: Optional[TranscribeConfig] = None,
    ) -> TranscriptionReturnType:
```


the `audio` is only limited to a list of strings, in which this case a path to the audio files.

That's a type annotation, it has no effect on the actual execution of code. That type annotation is wrong. But the code is going to execute RNNT with tensor just fine cause the underling model supports it 

thanks for the quick reply again, maybe I should try again!

One last question, is the Code Refactoring for ASR applied on 1.23.0 ??

No, it will be the next major version release 

any estimation of release date yet??

It's already in the main branch, you can try it out there. However, the next release will be probably at the end of the month. But it will probably be a prerelease 

thx, I tried with main branch and it works smooth like butter!
keep up the great work!!!