TypeError when I run eval_beamsearch_ngram.py
State: closed | Pull request: no

Hi! I encountered a problem during inference of Citrinet-1024 with a language model. When I run [eval_beamsearch_ngram.py](https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram.py), output is :

```
Traceback (most recent call last):
  File "eval_beamsearch_ngram.py", line 347, in <module>
    main()
  File "eval_beamsearch_ngram.py", line 256, in main
    all_probs = [kenlm_utils.softmax(logits) for logits in all_logits]
  File "eval_beamsearch_ngram.py", line 256, in <listcomp>
    all_probs = [kenlm_utils.softmax(logits) for logits in all_logits]
  File "/home/username/workspace/nemo_asr/scripts/kenlm_utils.py", line 39, in softmax
    print(x - np.max(x))
  File "<__array_function__ internals>", line 6, in amax
  File "/home/username/anaconda3/envs/nemo/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2706, in amax
    keepdims=keepdims, initial=initial, where=where)
  File "/home/username/anaconda3/envs/nemo/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 85, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
TypeError: max() received an invalid combination of arguments - got (out=NoneType, axis=NoneType, ), but expected one of:
 * ()
 * (name dim, bool keepdim)
      didn't match because some of the keywords were incorrect: out, axis
 * (Tensor other)
 * (int dim, bool keepdim)
      didn't match because some of the keywords were incorrect: out, axis
```

So I modified kenlm_utils/softmax function as

```
def softmax(x, to_cpu=False):
    if to_cpu:
        x = x.cpu().numpy()
    e = np.exp(x - np.max(x))
    return e / e.sum(axis=-1).reshape([x.shape[0], 1])
```

and call with to_cpu=True in [eval_beamsearch_ngram.py](https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram.py). The problem is solved. I just want to inform people who faced the same problem.

@VahidooX fyi

The transcribe method used to return Tensors, but now it returns numpy arrays. So softmax does not need the conversion.  I think you are using an older version of NeMo with the old transcribe method.

@EmreOzkose 
Hi!

Did you try to run eval_beamsearch_ngram.py from inside of Docker container or without docker?

I'm trying to use docker image nvcr.io/nvidia/nemo:1.0.0rc1 and catch some dependencies errors.
`File "eval_beamsearch_ngram.py", line 46, in <module>
    import kenlm_utils
ModuleNotFoundError: No module named 'kenlm_utils'
`
And no luck to find related pip package or any mentions in code of Nemo.
Only importing of kenlm_utils (except of two import lines).

No, I didn't use Docker image.

> No, I didn't use Docker image.

Thanks for the info.
Then I'll try to create new issue :)

That kenlm_utils is in the same folder as the script. Have you tried running the script from its folder?

> That kenlm_utils is in the same folder as the script. Have you tried running the script from its folder?

Unfortunately don't have folder kenlm_utils there.
There are only two files: _create_tarred_transformer_lm_dataset.py_ and _install_ctc_decoders.sh_
`root@dmzubr-All-Series:/quartz/nemo_test# ls /workspace/nemo/scripts/asr_language_modelling
create_tarred_transformer_lm_dataset.py  install_ctc_decoders.sh
`

It looks like you are not using the latest main branch of nemo. The eval_beamsearch_ngram.py script is a part of the main branch and would be in the next stable release. Please take a look here:
https://github.com/NVIDIA/NeMo/tree/main/scripts/asr_language_modeling/ngram_lm

You may use the main branch, or just copy these the missing files along with your script. You may also use train_kenlm.py to train the N-gram models. The documentation of these scripts can be found here: https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/asr_language_modeling.html

Great thanks!
I'll try to get the missing files and check again.
Then will feedback here.

Thanks again! :)

upd: Finally no errors.
@VahidooX 

if you are using older versions of nemo then you will be facing above error. 

 update the kenlm utils softmax function as below:
`
def softmax(X, to_cpu=False):
 try:
        e = np.exp(X - np.max(X))
        return e / e.sum(axis=-1).reshape([X.shape[0], 1])
    except:
        X = X.cpu().numpy()
        e = np.exp(X - np.max(X))
        return e / e.sum(axis=-1).reshape([X.shape[0], 1])
`