canary-1b not proccessing json files correctly
State: closed | Pull request: no
Labels: name: bug

name: stale

**Describe the bug**
This bug occurs when I try to use the canary-1b model, it is not processing the json file correctly

**Steps/Code to reproduce bug**
This is my  code
```
from nemo.collections.asr.models import EncDecMultiTaskModel

# load model
canary_model = EncDecMultiTaskModel.from_pretrained('nvidia/canary-1b')

predicted_text = canary_model.transcribe(
    "transcribe_manifest.json",
    batch_size=16,  # batch size to run the inference with
)
```
transcribe_manifest.json
```
{
    "audio_filepath": "/home/bot/roarai/rag/scraper/Scrape_vid/transcibe_manifest.json",
    "duration": 40.0,
    "taskname": "asr",
    "source_lang": "en",
    "target_lang": "en",
    "pnc": "yes"
}
```
Error
```
[NeMo I 2024-02-27 15:52:11 nemo_logging:381] PADDING: 0
[NeMo I 2024-02-27 15:52:15 nemo_logging:381] Model EncDecMultiTaskModel was successfully restored from /home/bot/.cache/huggingface/hub/models--nvidia--canary-1b/snapshots/5113160c7a3e52a7e5de1d904810d7aeabe6135b/canary-1b.nemo.
Traceback (most recent call last):
  File "/home/bot/roarai/rag/scraper/Scrape_vid/canary.py", line 12, in <module>
    predicted_text = canary_model.transcribe(
                     ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/site-packages/nemo/collections/asr/models/aed_multitask_models.py", line 406, in transcribe
    audio = manifest_utils.read_manifest(manifest_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/site-packages/nemo/collections/asr/parts/utils/manifest_utils.py", line 480, in read_manifest
    item = json.loads(line)
           ^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bot/anaconda3/envs/rag/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
```




Hmm can you try to see if you remove trailing new line in your manifest file, and also try to see if removing the json indent helps ?

@titu1994 I am experiencing much the same issue on my end. I tried as such and had no luck. I get the same JSON error as the one posted above. 
when doing as suggested by removing the JSON indent and trailing lines. I get the following error:
"json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 171 (char 170)"

When directly fixing the JSON errors, I end up with this error that suggests it expects the JSON to have an 'answer' key somewhere?!? : 

"-packages/nemo/collections/common/data/lhotse/nemo_adapters.py", line 84, in __iter__
    text=data[self.text_field],
KeyError: 'answer'
Transcribing: 0it [00:00, ?it/s]"

I want to run ASR, so expecting an 'answer' key is wonky. 




> text=data[self.text_field],
> KeyError: 'answer'

to fix this, please modify the input lines file to add 'answer' field. 
```
{
  "audio_filepath": "PathRemovedDueToPersonalName",  
  "duration": 30.0,  
  "taskname": "asr",  
  "source_lang": "en", 
  "target_lang": "en", 
  "pnc": 'yes', 
  "answer": 'na',
}
```
also, we recently updated `.transcribe` signature, so if you are using main branch 

> transcript = canary_model.transcribe(paths2audio_files="/home/pjstimac/NvidiaCanaryTest/transcribe_manifest.json", batch_size=16)

should be  updated to 
```transcript = canary_model.transcribe(audio="/home/pjstimac/NvidiaCanaryTest/transcribe_manifest.json", batch_size=16)```



@krishnacpuvvada No luck, unfortunately. Tried that exact format(with JSON & JSONL) with the updated transcript variable. Yields this error
 "  
  return func(*args, **kwargs)
TypeError: EncDecMultiTaskModel.transcribe() got an unexpected keyword argument 'audio
'"
 

> return func(*args, **kwargs)
> TypeError: EncDecMultiTaskModel.transcribe() got an unexpected keyword argument 'audio

can you verify that you are working with main branch of NeMo? the argument was updated from `paths2audio_files` to `audio` from r1.23 to main branch. 

Also as a sanity check, can you try the following on main branch

`trascript = canary_model.trascribe(audio=['path_to_file.wav']) # assuming path_to_file.wav is english audio. `


@krishnacpuvvada I'll double-check and update again when I'm home. But I was following the Nvidia tutorial and git-cloning the repo just last night, so I'm assuming it's the latest as of yesterday evening.  

The good news is I finally got basic inference working with three lines of code, and forgoing the JSON/JSONL manifest. 

" 
import nemo.collections.asr as nemo_asr
nemoasr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained("nvidia/canary-1b")
nemoasr_model.transcribe(['AudioClipDirectly.wav'])

"

To accomplish the transcription of a directory of audio clips, I wrote some Python code that loops through the directory and outputs the transcription results as JSON for my local model comparison app.

You can use this high level script to do inference with Canary as well - https://github.com/NVIDIA/NeMo/blob/main/examples/asr/transcribe_speech.py



@titu1994 I did try that as well per the Nvidia NeMo tutorial. Leading to the same exact issues I was running into elsewhere. 

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.