I could not achieve to run  eval_beamsearch_ngram.py script on my GPU.
State: closed | Pull request: no
Labels: name: bug

It is working on CPU but not working on my GPU. I tried Cuda 11.3, 11.8, 12.0 and Python 3.8, 3.9. It did not work. I got same error each time. 
This is how i run the code;
`python eval_beamsearch_ngram.py --nemo_model_file /media/mesut/Depo1/models/49/49.nemo --kenlm_model_file /media/mesut/Depo1/models/tbmmV8/final.nemo  --input_manifest /media/mesut/Depo1/works/NeMo/tools/asr_webapp/manifest_input.json --preds_output_folder out --beam_width 128 --beam_alpha 1 --beam_beta 0`

Error is something like that;
```
Reading Manifest /media/mesut/Depo1/works/NeMo/tools/asr_webapp/manifest_input.json ...: 1it [00:00, 10131.17it/s]
Transcribing:   0%|                                                                                                                                                                                               | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "eval_beamsearch_ngram.py", line 362, in <module>
    main()
  File "eval_beamsearch_ngram.py", line 269, in main
    all_logits = asr_model.transcribe(audio_file_paths, batch_size=args.acoustic_batch_size, logprobs=True)
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/app/NeMo/nemo/collections/asr/models/ctc_models.py", line 183, in transcribe
    logits, logits_len, greedy_predictions = self.forward(
  File "/app/NeMo/nemo/core/classes/common.py", line 1084, in __call__
    outputs = wrapped(*args, **kwargs)
  File "/app/NeMo/nemo/collections/asr/models/ctc_models.py", line 536, in forward
    processed_signal, processed_signal_length = self.preprocessor(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/app/NeMo/nemo/core/classes/common.py", line 1084, in __call__
    outputs = wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/app/NeMo/nemo/collections/asr/modules/audio_preprocessing.py", line 85, in forward
    processed_signal, processed_length = self.get_features(input_signal, length)
  File "/app/NeMo/nemo/collections/asr/modules/audio_preprocessing.py", line 268, in get_features
    return self.featurizer(input_signal, length)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/app/NeMo/nemo/collections/asr/parts/preprocessing/features.py", line 350, in forward
    x = self.stft(x)
  File "/app/NeMo/nemo/collections/asr/parts/preprocessing/features.py", line 242, in <lambda>
    self.stft = lambda x: torch.stft(
  File "/usr/local/lib/python3.8/dist-packages/torch/functional.py", line 632, in stft
    return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore[attr-defined]
RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR

```

Ubuntu 20.04
GPU: Gigabyte Windforce 4090