The decoded sentence is empty in the Japanese data set
State: closed | Pull request: no

Hello,
I have used the JSUT (Japanese speech corpus of Saruwatari-lab., University of Tokyo) data set to train the ASR model. 
I set `normalize_transcript` parameter of train_ds and validation_ds to False.
```
params['model']['train_ds']['manifest_filepath'] = train_manifest
params['model']['train_ds']['normalize_transcripts'] = False
params['model']['train_ds']['labels'] = labels
params['model']['validation_ds']['manifest_filepath'] = test_manifest
params['model']['validation_ds']['normalize_transcripts'] = False
params['model']['validation_ds']['labels'] = labels
params['model']['preprocessor']['params']['stft_conv'] = True
params['model']['decoder']['params']['num_classes'] = len(labels)
params['model']['decoder']['params']['vocabulary'] = labels
params['labels'] = labels
```
However, the "decoded" of the result of the `trainer.fit()` is empty after the second epoch.
```
[NeMo I 2021-02-25 12:33:22 modelPT:597] Optimizer config = Novograd (
    Parameter Group 0
        amsgrad: False
        betas: [0.8, 0.5]
        eps: 1e-08
        grad_averaging: False
        lr: 0.01
        weight_decay: 0.001
    )
[NeMo I 2021-02-25 12:33:22 lr_scheduler:562] Scheduler "<nemo.core.optim.lr_scheduler.CosineAnnealing object at 0x7f35252f6750>" 
    will be used during training (effective maximum steps = 1398) - 
    Parameters : 
    (warmup_steps: null
    warmup_ratio: null
    min_lr: 0.0
    last_epoch: -1
    max_steps: 1398
    )

  | Name              | Type                              | Params
------------------------------------------------------------------------
0 | preprocessor      | AudioToMelSpectrogramPreprocessor | 0     
1 | encoder           | ConvASREncoder                    | 1.2 M 
2 | decoder           | ConvASRDecoder                    | 2.8 M 
3 | loss              | CTCLoss                           | 0     
4 | spec_augmentation | SpectrogramAugmentation           | 0     
5 | _wer              | WER                               | 0     
[NeMo I 2021-02-25 12:33:24 wer:159] 
    
[NeMo I 2021-02-25 12:33:24 wer:160] reference:演奏会の切符は当事務所で販売しています
[NeMo I 2021-02-25 12:33:24 wer:161] decoded  :平溝平溝平溝捕溝平溝平溝平情溝捕溝捕平捕平捕平溝平捕溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平溝捕溝捕平溝平溝平溝捕溝平溝平溝平溝平溝捕溝捕溝平
[NeMo I 2021-02-25 12:33:24 wer:159] 
    
[NeMo I 2021-02-25 12:33:24 wer:160] reference:鳥だったらすぐそこに飛んでいくのに
[NeMo I 2021-02-25 12:33:24 wer:161] decoded  :平溝捕溝捕溝平溝平情溝平溝平溝平捕平溝情溝平溝平溝平溝平溝平溝平溝平溝捕平溝捕溝平溝平溝平溝平溝平溝平溝平情溝平溝平溝平溝平捕溝平溝平情溝捕溝捕平溝平溝平情溝平溝平捕溝捕溝捕溝捕溝捕溝平溝平溝平溝平溝捕平溝平捕溝捕溝平溝捕溝捕溝平溝平溝平溝平溝平溝平溝平溝平溝平溝平捕溝捕溝平
Epoch 2: 100%
474/474 [02:35<00:00, 3.06it/s, loss=80.696, v_num=1]
[NeMo I 2021-02-25 12:33:41 wer:159] 
    
[NeMo I 2021-02-25 12:33:41 wer:160] reference:
[NeMo I 2021-02-25 12:33:41 wer:161] decoded  :
[NeMo I 2021-02-25 12:33:57 wer:159] 
    
[NeMo I 2021-02-25 12:33:57 wer:160] reference:孫策の死後はやや無気力感を漂わせたが頭の中では自身の周りで起きた運命や世の中の流れについて終わる事のない自問を繰り返していた
[NeMo I 2021-02-25 12:33:57 wer:161] decoded  :
[NeMo I 2021-02-25 12:34:14 wer:159] 
    
[NeMo I 2021-02-25 12:34:14 wer:160] reference:学識の点で彼は彼の友人のだれよりも上です
[NeMo I 2021-02-25 12:34:14 wer:161] decoded  :
[NeMo I 2021-02-25 12:34:30 wer:159]
```

Any help would be appreciated.

**Environment overview (please complete the following information)**
 - Environment location: Collab (https://colab.research.google.com/drive/1kPb1p-3IRfjCu1Hu_kTRHwSOsWmSvA7V?usp=sharing)
 - Method of NeMo install: pip install from github
 - NeMo version: r1.0.0b3


The training loss is near 150~ after two epochs. You need to train longer so that loss drops to around 50-80 or lower before tokens actually appear. 

The first phase of CTC learning is to learn to predict all blank tokens. 

Thank you very much!
I am going to train more longer.

I got a better result after 1200 epoch.
Thank you very much!
```
[NeMo I 2021-02-28 03:43:45 wer:160] reference:簡易ベッドの組み立て方を教えてもらえますか
[NeMo I 2021-02-28 03:43:45 wer:161] decoded  :ーイベットのみたて方をを教えてもらいますか
[NeMo I 2021-02-28 03:43:46 wer:159] 
[NeMo I 2021-02-28 03:43:46 wer:160] reference:また近いうちにお目にかかりましょう
[NeMo I 2021-02-28 03:43:46 wer:161] decoded  :まただいうちにおめにかかりましょう
[NeMo I 2021-02-28 03:43:46 wer:159] 
[NeMo I 2021-02-28 03:43:46 wer:160] reference:夫からもくれぐれもよろしくとのことです
[NeMo I 2021-02-28 03:43:46 wer:161] decoded  :夫からもれぐれもよろしくとのことです
[NeMo I 2021-02-28 03:44:00 wer:159] 
[NeMo I 2021-02-28 03:44:00 wer:160] reference:車や飛行機ではいつも安全ベルトを締めることが法律で決められています
[NeMo I 2021-02-28 03:44:00 wer:161] decoded  :車や飛行機ではいつも安全ベルトを締めることが法律で決められています
[NeMo I 2021-02-28 03:44:14 wer:159] 
[NeMo I 2021-02-28 03:44:14 wer:160] reference:現在の滑走を目的としたスキーブーツは硬いプラスチックシェルと柔らかいインナーブーツからなる
[NeMo I 2021-02-28 03:44:14 wer:161] decoded  :現在の滑走を目的としたスキーブーツは硬いプラスチックシェルと柔らかいインナブーツからなる
[NeMo I 2021-02-28 03:44:28 wer:159] 
```