Can't run basic inference
State: closed | Pull request: no

Hi, all. These days I want to use 'canary-1b' to try ASR function, but when I followed this[ tutorial](https://developer.nvidia.com/blog/new-standard-for-speech-recognition-and-translation-from-the-nvidia-nemo-canary-model/), I always got **AssertionError: Expected MonoCut**. Here is the whole output.

(base) root@autodl-container-02954198e9-e04d5d74:~/autodl-tmp/canary-1b-test# python version_test.py
[NeMo I 2024-09-02 18:34:48 mixins:196] _setup_tokenizer: detected an aggregate tokenizer
[NeMo I 2024-09-02 18:34:48 mixins:330] Tokenizer SentencePieceTokenizer initialized with 32 tokens
[NeMo I 2024-09-02 18:34:48 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:34:48 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:34:48 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:34:48 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:34:48 aggregate_tokenizer:72] Aggregate vocab size: 4128
[NeMo W 2024-09-02 18:34:49 modelPT:165] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
    Train config : 
    tarred_audio_filepaths: null
    manifest_filepath: null
    sample_rate: 16000
    shuffle: true
    batch_size: null
    num_workers: 8
    use_lhotse: true
    max_duration: 40
    pin_memory: true
    use_bucketing: false
    bucket_duration_bins: null
    num_buckets: 1
    text_field: answer
    lang_field: target_lang
    batch_duration: 360
    quadratic_duration: 15
    bucket_buffer_size: 20000
    shuffle_buffer_size: 10000
    
[NeMo W 2024-09-02 18:34:49 modelPT:172] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s). 
    Validation config : 
    manifest_filepath: null
    sample_rate: 16000
    batch_size: 8
    shuffle: false
    num_workers: 0
    pin_memory: true
    tarred_audio_filepaths: null
    use_lhotse: true
    text_field: answer
    lang_field: target_lang
    use_bucketing: false
    
[NeMo W 2024-09-02 18:34:49 modelPT:178] Please call the ModelPT.setup_test_data() or ModelPT.setup_multiple_test_data() method and provide a valid configuration file to setup the test data loader(s).
    Test config : 
    manifest_filepath: null
    sample_rate: 16000
    batch_size: 32
    shuffle: false
    num_workers: 0
    pin_memory: true
    tarred_audio_filepaths: null
    use_lhotse: true
    text_field: answer
    lang_field: target_lang
    use_bucketing: false
    
[NeMo I 2024-09-02 18:34:49 features:289] PADDING: 0
[NeMo W 2024-09-02 18:35:06 nemo_logging:349] /root/miniconda3/lib/python3.10/site-packages/nemo/core/connectors/save_restore_connector.py:571: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
      return torch.load(model_weights, map_location='cpu')
    
[NeMo I 2024-09-02 18:35:13 save_restore_connector:249] Model EncDecMultiTaskModel was successfully restored from /root/.cache/huggingface/hub/models--nvidia--canary-1b/snapshots/dd32c0c709e2bfc79f583e16b9df4b3a160f7e86/canary-1b.nemo.
Traceback (most recent call last):
  File "/root/autodl-tmp/canary-1b-test/version_test.py", line 3, in <module>
    transcript = canary_model.transcribe(audio=["segment_0.wav"])
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
TypeError: EncDecMultiTaskModel.transcribe() got an unexpected keyword argument 'audio'
(base) root@autodl-container-02954198e9-e04d5d74:~/autodl-tmp/canary-1b-test# python version_test.py
[NeMo I 2024-09-02 18:36:14 mixins:196] _setup_tokenizer: detected an aggregate tokenizer
[NeMo I 2024-09-02 18:36:14 mixins:330] Tokenizer SentencePieceTokenizer initialized with 32 tokens
[NeMo I 2024-09-02 18:36:14 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:36:14 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:36:14 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:36:14 mixins:330] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo I 2024-09-02 18:36:14 aggregate_tokenizer:72] Aggregate vocab size: 4128
[NeMo W 2024-09-02 18:36:15 modelPT:165] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
    Train config : 
    tarred_audio_filepaths: null
    manifest_filepath: null
    sample_rate: 16000
    shuffle: true
    batch_size: null
    num_workers: 8
    use_lhotse: true
    max_duration: 40
    pin_memory: true
    use_bucketing: false
    bucket_duration_bins: null
    num_buckets: 1
    text_field: answer
    lang_field: target_lang
    batch_duration: 360
    quadratic_duration: 15
    bucket_buffer_size: 20000
    shuffle_buffer_size: 10000
    
[NeMo W 2024-09-02 18:36:15 modelPT:172] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s). 
    Validation config : 
    manifest_filepath: null
    sample_rate: 16000
    batch_size: 8
    shuffle: false
    num_workers: 0
    pin_memory: true
    tarred_audio_filepaths: null
    use_lhotse: true
    text_field: answer
    lang_field: target_lang
    use_bucketing: false
    
[NeMo W 2024-09-02 18:36:15 modelPT:178] Please call the ModelPT.setup_test_data() or ModelPT.setup_multiple_test_data() method and provide a valid configuration file to setup the test data loader(s).
    Test config : 
    manifest_filepath: null
    sample_rate: 16000
    batch_size: 32
    shuffle: false
    num_workers: 0
    pin_memory: true
    tarred_audio_filepaths: null
    use_lhotse: true
    text_field: answer
    lang_field: target_lang
    use_bucketing: false
    
[NeMo I 2024-09-02 18:36:15 features:289] PADDING: 0
[NeMo W 2024-09-02 18:36:31 nemo_logging:349] /root/miniconda3/lib/python3.10/site-packages/nemo/core/connectors/save_restore_connector.py:571: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
      return torch.load(model_weights, map_location='cpu')
    
[NeMo I 2024-09-02 18:36:37 save_restore_connector:249] Model EncDecMultiTaskModel was successfully restored from /root/.cache/huggingface/hub/models--nvidia--canary-1b/snapshots/dd32c0c709e2bfc79f583e16b9df4b3a160f7e86/canary-1b.nemo.
Transcribing: 0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/root/autodl-tmp/canary-1b-test/version_test.py", line 3, in <module>
    transcript = canary_model.transcribe(["segment_0.wav"])
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/nemo/collections/asr/models/aed_multitask_models.py", line 472, in transcribe
    for test_batch in tqdm(temporary_datalayer, desc="Transcribing", disable=not verbose):
  File "/root/miniconda3/lib/python3.10/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
    data = self._next_data()
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1344, in _next_data
    return self._process_data(data)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1370, in _process_data
    data.reraise()
  File "/root/miniconda3/lib/python3.10/site-packages/torch/_utils.py", line 706, in reraise
    raise exception
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
    data = self.dataset[possibly_batched_index]
  File "/root/miniconda3/lib/python3.10/site-packages/nemo/collections/asr/data/audio_to_text_lhotse_prompted.py", line 55, in __getitem__
    tokens = self.prompt_format_fn(cuts, self.tokenizer)
  File "/root/miniconda3/lib/python3.10/site-packages/nemo/collections/asr/data/audio_to_text_lhotse_prompted.py", line 127, in canary
    assert isinstance(cut, MonoCut), "Expected MonoCut."
AssertionError: Expected MonoCut.


and here is my code:

`from nemo.collections.asr.models import EncDecMultiTaskModel
canary_model = EncDecMultiTaskModel.from_pretrained('nvidia/canary-1b')
transcript = canary_model.transcribe(["segment_0.wav"])
print(transcript)`

Hi, today I changed the other two audios by using the same code and it worked. The duration of these two audios are 29.131020408163266s and 17.04204081632653s respectively. And the duration of the failed one is 29.861043083900228s. All of these three audios are WAV format. I don't know what happened, could anyone please help me deal with this? 

I just used ffprobe to check the meta info, it shows the failed one is stereo, the other two are mono. I think that's the reason. And I tried converting stereo audio to mono, then use the same code, it worked.